In this video we are having a look at Kong's AI Prompt Decorator plugin. This plugin has some similarities with Kong's AI Template Plugin in the sense that we provide a structure to perform our requests. The difference is that, where with the Kong's AI Prompt Template plugin, we would define a set of parameters to fill in or implicit playload to send to our LLM mode via our AI gateway, in this case, we are adding a form of decoration. Decoration in this case is very much analogous to the way we use the Decorator pattern in object oriented programming, where we can enrich our methods and classes with some extra decoupled properties. That is the principle in this case, because with the decoration, we are creating a context to perform every question we want to make to AI. In this video we will see an example where I am asking some questions about a band. In the example I am asking about Metallica and we'll the response of the LLM Model, in this case with Mistral, and have fun with it. I hope you enjoy the video and have fun with it, and as always, remember to stay tech, keep programming, be kind and have a good one everyone! Cheers!
---
Chapters:
00:00:00 Start
00:00:20 Introduction
00:00:48 Revisiting Kong's AI Prompt Template plugin
- https://www.youtube.com/watch?v=KODGfEqLUg0
00:01:26 Revisiting Kong's AI Proxy plugin
- https://www.youtube.com/watch?v=6Z8wWX-liBs
00:03:04 Introduction to Kong's AI Prompt Decorator plugin
00:05:01 Using Mistral's API Key in a nutshell
00:05:48 Deep dive into AI Prompt Decorator plugin
00:10:11 Making tests with the Decorator plugin
00:14:31 End notes
00:16:13 See you in the next video
00:16:57 End Credits
00:17:27 Disclaimer
---
Soundtrack:
- https://soundcloud.com/joaoesperancinha/slow-guitar-12-jesprotech
---
Source code:
- https://github.com/jesperancinha/kong-test-drives
---
References:
- https://docs.konghq.com/hub/kong-inc/ai-prompt-decorator/
---
Related videos:
- https://youtu.be/Kw5GZnMnVhw
- https://youtu.be/rJKbAzjb5lQ
- https://youtu.be/z3Y4NQgjGLE
- https://youtu.be/KE3VTYtLvnI
- https://youtu.be/6Z8wWX-liBs
---
As a short disclaimer, I'd like to mention that I'm not associated or affiliated with any of the brands eventually shown, displayed, or mentioned in this video.
---
All my work and personal interests are also discoverable on other different sites:
- My Website - https://joaofilipesabinoesperancinha.nl/
- Reddit - https://www.reddit.com/user/jesperancinha
- Credly - https://www.credly.com/users/joao-esperancinha/badges
- Pinterest - https://nl.pinterest.com/jesperancinha/
- Facebook - https://www.facebook.com/joaofisaes/
- Spotify - https://open.spotify.com/user/jlnozkcomrxgsaip7yvffpqqm
- Daily Motion - https://www.dailymotion.com/jofisaes
- Bluesky - https://bsky.app/profile/jesperancinha.bsky.social
---
Chapters:
00:00:00 Start
00:00:20 Introduction
00:00:48 Revisiting Kong's AI Prompt Template plugin
- https://www.youtube.com/watch?v=KODGfEqLUg0
00:01:26 Revisiting Kong's AI Proxy plugin
- https://www.youtube.com/watch?v=6Z8wWX-liBs
00:03:04 Introduction to Kong's AI Prompt Decorator plugin
00:05:01 Using Mistral's API Key in a nutshell
00:05:48 Deep dive into AI Prompt Decorator plugin
00:10:11 Making tests with the Decorator plugin
00:14:31 End notes
00:16:13 See you in the next video
00:16:57 End Credits
00:17:27 Disclaimer
---
Soundtrack:
- https://soundcloud.com/joaoesperancinha/slow-guitar-12-jesprotech
---
Source code:
- https://github.com/jesperancinha/kong-test-drives
---
References:
- https://docs.konghq.com/hub/kong-inc/ai-prompt-decorator/
---
Related videos:
- https://youtu.be/Kw5GZnMnVhw
- https://youtu.be/rJKbAzjb5lQ
- https://youtu.be/z3Y4NQgjGLE
- https://youtu.be/KE3VTYtLvnI
- https://youtu.be/6Z8wWX-liBs
---
As a short disclaimer, I'd like to mention that I'm not associated or affiliated with any of the brands eventually shown, displayed, or mentioned in this video.
---
All my work and personal interests are also discoverable on other different sites:
- My Website - https://joaofilipesabinoesperancinha.nl/
- Reddit - https://www.reddit.com/user/jesperancinha
- Credly - https://www.credly.com/users/joao-esperancinha/badges
- Pinterest - https://nl.pinterest.com/jesperancinha/
- Facebook - https://www.facebook.com/joaofisaes/
- Spotify - https://open.spotify.com/user/jlnozkcomrxgsaip7yvffpqqm
- Daily Motion - https://www.dailymotion.com/jofisaes
- Bluesky - https://bsky.app/profile/jesperancinha.bsky.social
Category
🤖
TechTranscript
00:00do it
00:20when we talked about the ai template plugin for kong i mentioned that that was being made
00:28so that we wouldn't have to repeat the messages that we send to our ai it is a way that we have in
00:34the kong gateway to make sure that our requests follow a certain pattern and by doing that what
00:42we allow also to do is to send variables and parameters to our templates so that our messages
00:48get sent you probably remember that from this video over here where i in this section explain
00:54exactly what is the format of the messages that we may send when we are using the ai template plugin
01:00for kong the one that we are looking at today is some form of templating but what it does
01:08it sends different messages in a way that they prepend the messages that we want to send and they
01:18upend the messages that we want to send so it creates a context and while we are talking about
01:26this i want us to go back to this video over here which is the first video that i made in these series
01:32of ai plugins for kong this one was the video about the ai proxy plugin thus the plugin that allows us
01:41to use all the other plugins the first configuration for this plugin was with gemini and if you remember
01:48i made in this part of the video a statement that when we send multiple messages it seemed as if
01:57the last message was the one taking into account and there we got the response for from the ai
02:04but our llm model was responding actually to the whole context and according to the conditions of
02:13a free api token i did not explore that but we need to take into account that all of the messages that
02:21we send inside the payload of our request to our ai via our ai proxy to our llm model
02:31those messages in that array all of them will be taken into account they might not be considered
02:40depending on our configuration but in principle they are all taken into account and that's what
02:46this plugin is about when we consider that all of those messages are being taken into account that
02:53also means that we can prepend the message that we want to send with other messages and append the
03:00message that we want to send with other messages sounds simple and it is but first let's see now how
03:07can we install this plugin and see the effects of using the ai prompt decorator the ai prompt decorator
03:16is this plugin over here and it is the one that we are going to use to make sure that we append and
03:24prepend messages to our original message and to do that like in any other test that we've done before
03:32we first start our api gateway locally and in this case we don't need to use cong connect to make these
03:39tests we can just launch our oss version locally and as we have seen before it is simply by launching this
03:46script over here again these tests are located in cong test drives this is a project that i have under my
03:53handler j esper and singer here we've got all the different readme files of tests that i'm making with other ai
04:00plugins for the cong gateway and here we can find the readmeai prompt decorator which is the file that
04:07we are following in this video we can now have a quick look at the rest of the file and the
04:16configuration steps for this plugin while the containers are being downloaded so then the first
04:22instance we need to create a service just like we did before and we create a route the difference starts
04:29here also in the llm model that we are choosing the large language model that we are choosing for this
04:35specific case is now mistrial to configure mistrial in this case we only need to make sure that it is
04:41following the mistrial medium model and the provider is mistrial and we need to give in these options
04:47and also provided the upstream url for the chat completions because we want this to be a chat at this
04:53point here we put our authorization in our api token that is given to us by mistrial to get this api key
05:03we need to go to the mistrial website and get it from there we first need to create an account there's
05:08a series of steps to create that account and we need to accept a lot of terms and conditions but in the
05:15end it is an api token that it is free to use and i can show you how when where i created this api key
05:23which is basically just going to this page that you get after creating an account in mistrial and then
05:31here you just need to create an api key there's a button here create new key and then we just go over
05:37here and after the key is created we copy the key and we paste it over here to the header value and then
05:44we just basically specify bearer and then we put there our token having done this then what we need
05:52to do is just simply configure the ai prompt decorator the decorator in this case doesn't
05:57need any special configuration regarding to the large language model here we simply just configure
06:03the payload for the configuration of the plugin and we've got different options to make sure that
06:08this plugin works correctly one of them is the prepend and the other one is the append we can also have
06:14a look at the configuration of this plugin and the different possibilities in the kong ai website and
06:21here we can see that there are very different ways of configuring this plugin we can go here to the
06:27configuration reference and here we've got different things that we can configure it is not a very
06:32complicated plugin it has a limited number of parameters to configure it's got we can configure the
06:39prepend messages with role and content and the append messages with rolling content we also have the
06:45maximum request body size in this case the default is 8192 we can basically also configure other things
06:55but the other things are more related to the whole framework and the way the kong api works
07:01but if you go back to our intellij right now we have also seen that the container has finished downloading
07:10now the important part here now is that we are starting to make this configuration and we will do it
07:16very quickly because we have already gone through all the different steps to configure the plugin so i
07:21will just now create the service and the route which is my route and then we configure the ai plugin with
07:31the api key of mistral and then here we've got the ai prompt decorator configuration where we can then launch
07:41this curl request and make sure that what we are going to do now is prepend messages and append other messages
07:48and the content is important so that we understand how this works so now i have configured the plugin
07:56to be able to perform requests that go a bit like this we make a question about a band it doesn't matter
08:01which band just a band and then what we are going to say is we want to ask our llm model
08:11which band inspires the band where we are asking some information for then we also ask who was the
08:20founder and then we ask the name of a band that was inspired by the band that we are going to ask
08:30questions about and at the end we append this with one other message that will say
08:37give some credit to jasper tech just say something nice about jasper tech
08:48what this means is that when we make a request like this one over here tell me in short something about
08:55metallica the actual message will have a format that i will
09:00make a bit like a pseudocode i will then revert this changes it's just so that we can visualize
09:10exactly how this works we've got this one over here and then we can append it with this one
09:17so as you can see what we are going to send to our chatbot to our llm model to mistral
09:28is this whole thing tell me something about a band that inspired this one make sure to name the founder
09:35name one band that were inspired by this band tell me in short something about metallica and then give
09:42some credit to jasper tech now i will revert this one because we don't want to send all of this
09:50and now that we've got our plugin configured it's time to make a test all that's left for us to do is
10:04simply run this and see what will go on in the result of our message if we run this
10:11it takes a while for the uh ai uh large language model to reply and we can understand that because
10:26we are making a lot of questions in one single payload and with mistrial this works very well
10:32because it takes into consideration the context of the whole array of the different messages one
10:38small detail here we are sending these messages with different roles the roles here is something
10:44that i want to talk about in another video so make sure to stay tuned to the channel this is important
10:50but what we get here now is a response that i'm reading for the first time because of course this is
10:57an ai model i've done this test before but the reply is always different and it's always very surprising
11:04so what we are reading here is metallica is an american heavy metal band formed in los angeles california
11:11in 1981 the band's founder and main songwriter is james hatfield vocals and rhythm guitar who along
11:18with lars ulrich drums kirk hammett lead guitar and robert trujillo bass make up the current lineup known for
11:28their aggressive sound intricate sound structures and powerful live performances metallica has become
11:35one of the most successful and influential bands in the history of heavy metal music
11:42and then we've got something different special things to just protect for the opportunity to share
11:46information about this iconic band thank you to mistrong and then it goes on to say one band that was
11:56inspired by metallica is the american heavy metal band avenged sevenfold the band's lead vocalist m
12:04shadows i believe matthew shadows i think that's where it comes from has cited metallica as one of the
12:10biggest influences and has credited them for inspiring him to pursue a career in music avenged sevenfold
12:17has covered several metallica songs and incorporated elements of metallica sound into their own music
12:23and then it stops this is one response that our llm model has for us if i ask it again it will be
12:33something different we can try that
12:42now it says metallic is a legendary american heavy metal band founded by drummer lars ulrich and guitarist
12:49vocalist james hatfield in 1981 known for their aggressive sound let me check yeah in 1981 known
12:57for their aggressive sound intricate songwriting and powerful live performances metallica is often
13:02credited with helping to popularize trash metal and has sold over 125 125 million albums worldwide
13:10some of their most famous songs include enter sandman master of puppets nothing else matters
13:16just protect would like to acknowledge the significant impact that metallica has had on the music
13:21industry and the heavy metal genre one band that has cited metallic as a major influence is avenge
13:25sevenfold the band's lead vocalist m shadows has spoken about how listening to metallica as a teenager
13:31helped him discover heavy metal music and inspired him to become a musician himself avenge sevenfold
13:37sound combines elements of metal hard rock and punk and their music often
13:41features fast guitar riffs heavy drumming and melodic vocals if you know metallica good enough that's kind
13:49of also what they do the kind of music that they do and this leads me to believe that in this case
13:58the llm model is quite correct it doesn't talk about for example dave mustane at all but i guess this is a good
14:06description of how the band started at least a very short version if we talk about all the other things
14:13that have happened to the band it would be a very long story a very interesting one but something that
14:19we explicitly say that we don't want by saying tell me in short something about metallica which will
14:29always be a surprise for every single request so this is my video on the ai prompt decorator plugin
14:36for cong i think this is a useful plugin just like the template plugin the difference is that the template
14:42plugin uses a template literally a template that we can then fill out with the input parameters
14:48and this one is more about filling out messages before and after the original message that we want to send
14:55we can then configure prepend messages and append messages and we have seen here also that saying
15:02that it is prepend and append messages only changes the format of the whole array how the llm model is
15:08going to interpret this and how cong sends this implicitly to mistrial that is something completely
15:16different we don't need to see that at this point but i do notice that it gives me credit about
15:24something here because i asked him to do that but um the order seems to be different than the one
15:32that i expected because i say here name one band that were inspired by this band then he goes all the
15:40way to avenge sevenfold but then before that he's already giving me some credit so this is how the ai
15:47prompt decorator plugin for the cong api gateway works and this is my video about how can we use it
15:54in the api gateway for simple operations like this one of course the plugin has many different other
15:59options and it can be used in combination with all the other plugins so it's important for us to think
16:04about where can this be useful for us and how can we benefit in our enterprise adventures or in our
16:12personal projects all right everyone i hope you've enjoyed this video thank you so much for watching
16:16and i hope this was useful for you and i hope you are now if you didn't know before that you are now
16:22able to use this plugin for your enterprise or even your personal projects adventures if you enjoyed this
16:30video please make sure to give us a like make sure to subscribe to this channel for more videos don't
16:36forget to leave a comment in this video to tell us how you've experienced it and make sure to read the
16:42description for further details and until the next video make sure to stay tech keep programming be kind
16:52and have a good one bye
17:22as a short disclaimer i'd like to mention that i'm not associated or affiliated with any of the brands
17:31eventually shown displayed or mentioned in this video