In the previous post, we talked about the implementation details of how the demo app works & how to set up Semantic Kernel, with the StepwisePlanner and Azure OpenAI. Now that you understand the code, let’s look at the demo application.
Here is the link to the GitHub repo.
Use case
As a reminder, we are building a customer support chatbot for our fictional outdoor sporting equipment company. We want to be able to answer common customer support questions by utilizing our internal data sources.
Example query
Will my sleeping bag work for my trip to Patagonia next month?
Fictional data sources
Demo app
This demo app allows the user to make a request. That request is then run through the Sematic Kernel StepwisePlanner. The plan that is generated is then executed and the response is sent back to the user.
In this example, we are explicitly showing the planner steps to demonstrate the execution plan. In the real world, you wouldn’t show the execution plan (since this reveals internal implementation details such as the data sources used, their inputs & responses, errors, etc).
We can see that the planner made a plan with 6 steps. Let’s go through each one in detail.
1. OrderHistoryPlugin.OrderHistoryLookup
Here are the thoughts and observations of the first OpenAI call. The planner recognizes that to answer the question, it needs to call numerous plugins that are registered (the ProductCatalogPlugin, the HistoricalWeatherLookupPlugin and the OrderHistoryPlugin). It makes the first action call to the OrderHistoryPlugin.OrderHistoryLookup function (passing in the username of the user that made the request). The result is a JSON document. That document includes the productID, which is the most relevant detail for calling the next step.
Note that I didn’t write any code to parse the results, didn’t write any code to define the order of operations, didn’t write any code to explicitly map the username that is stored in the context variables with the input parameter the function needed. Yes, they have the same name, but all the magic is in the Description attribute decorators that explain (in human-understandable terms) what the functions do and how to call them.
2. ProductCatalogPlugin.ProductCatalogItemLookup
Now that it knows the productID for the specific sleeping bag the user was interested in, it can look in the product specifications to find out the lowest supported temperature of the sleeping bag. Note that I didn’t write any code to directly do this mapping; OpenAI figured it out on its own based upon my descriptions of the input, output & function descriptions.
3. HistoricalWeatherLookupPlugin.HistoricalWeatherLookup – #1
Now that we have the specs for the sleeping bag (including the lowest supported temperature), we can figure out the lowest temperature to expect during our camping trip.
In this example, I explicitly wrote the sample HistoricalWeatherLookup API to fail if the wrong GPS coordinates are passed in (notice the -51.796253 & -72.302413) or the wrong date is passed in. While these coordinates are near “Patagonia”, I wanted the system to fail if it doesn’t use my custom LocationLookupPlugin to find the exact coordinates of the user’s specified location.
Notice that the planner was able to determine that it needed to increment the date of the month (these screenshots were created in November). The user asked for “next month”. Since we set a context variable with today’s date, OpenAI was able to deduce that it needed to increment the month.
The initial call to the HistoricalWeatherPlugin.HistoricalWeatherLookup failed. Let’s see what the Semantic Kernel StepwisePlanner & Azure OpenAI decides to do next.
4. LocationLookupPlugin.LocationLookup
Part of the magic of the StepwisePlanner is that it can try and call plugins and if there is an error, it can try and correct. In this case, the HistoricalWeatherLookup doesn’t return data if the GPS coordinates & the date are not correct.
The planner decided that it could use the LocationLookupPlugin.LocationLookup to find new GPS coordinates.
5. HistoricalWeatherLookupPlugin.HistoricalWeatherLookup – #2
Now that we have retrieved valid GPS coordinates, the StepwisePlanner is going to try again and call the HistoricalWeatherLookup plugin with different GPS coordinates (the ones it retrieved from the LocationLookupPlugin.
We now have the lowest expected temperature we can expect to experience for those GPS coordinates & that month.
6. Final answer
Now that we have all the needed information (lowest temperature expected & lowest supported temperature), OpenAI can finally answer the user’s question.
Conclusion
Again, the magic is that I (as a human) know the happy path through the system. I know the order of operations, the inputs & outputs. However, I don’t want to have to hard-code this access path, try and make “if/else”, “switch” or “regex” statements to try and pick the right path based upon the user’s question.
The Semantic Kernel StepwisePlanner & Azure OpenAI managed to figure everything out on its own (based upon my descriptions of what the plugins do).
Caveat emptor!
A key consideration of using any large language model & building it into your application is the non-deterministic nature of the responses. You cannot guarantee responses. All you can do is influence the response based upon the prompts.
The same is true for the StepwisePlanner and Azure OpenAI. You can’t guarantee that it will call your plugins in the right order, that it will pass in the right data, that it will parse the output correct or that it will glean the correct insight from the data returned.
You have to be ok with the fact that it is non-deterministic. If you have to have a deterministic response, you will end up hard-coding the execution path (even if you use Semantic Kernel).
In the next post, I’ll talk about some of the techniques I used to make the demo app easy to run & deploy.