From Questions to Insights: Unlocking the Power of ACS with AI

Using LLMs to identify the correct API codes for a user’s query involves several key steps and challenges. Initially, I used an LLM to extract the user’s intent, keywords, and location information from their query. This part was straightforward, as the LLM efficiently identified relevant terms and geographies. The real complexity arose when determining which API codes from the extensive database were most appropriate to answer the query.

With over 20,000 API codes covering various topics, I needed a strategy to narrow the potential matches. My first approach was to group codes by their concepts, but the 635 unique concepts were still too broad. To simplify, I manually categorized these concepts into 13 higher-level categories, such as Education, Income, and Transportation, which allowed me to exclude large sections of irrelevant codes early in the process.

I needed finer granularity to improve matching accuracy even with categories in place. I used the NLTK library to extract keywords from the API name, concept, and category descriptions to achieve this. This provided a foundational keyword list for each API code. To expand these lists, I leveraged an LLM to generate synonyms for each keyword in the context of the ACS database. For example, the database used “male,” so I added synonyms like “man” and “men.” Similarly, for “college,” I included terms like “university” and “secondary education.” This enhanced keyword matching significantly reduced the number of candidate API codes.

Despite these improvements, category and keyword matching alone sometimes left hundreds of potential matches for queries like “What is the Hispanic population of Michigan?”, which would return over 1,000 potential API codes.

To refine the results further, I employed a recursive LLM prompt strategy. I split the list of candidate codes into manageable “chunks” that fit within the LLM’s context window. Each chunk was processed by the LLM, which filtered out only those codes whose descriptions aligned with the user’s query. By repeating this process recursively, I reduced the list to a maximum of 30 codes.

One critical challenge I faced was the 45-second computation time limit of the Azure Function running my analysis. This constraint required optimizing the chunk size to balance context processing time, LLM latency, and compute limits. After testing, I found that a chunk size of 300 codes struck the optimal balance, allowing the system to process queries efficiently within the time limit.

This multi-layered approach—combining categories, expanded keyword matching, and recursive filtering with LLMs—enabled the system to accurately identify the best API codes for a wide variety of user queries while staying within performance constraints.