Hi,
We are in the process of migrating to the Freedom UI and as such have access to the AI capabilities, for which I am interested to learn from other users of their experiences.
Have your users and company found the use of AI of benefit and if so, what would you say would biggest benefits you have found?
During testing, we have found a very high average token consumption (between 20,000 and 50,000) and am concerned about how much each interaction 'costs' in tokens and that will limit our usage.
Any feedback would be appreciated.
thanks
Mark
Like
Hello,
Please refer to the following, below is an example of how many tokens each intent can use:
More information about tokenization can be found here:
1. https://learn.microsoft.com/en-us/dotnet/ai/conceptual/understanding-tokens
2. Actual visual representation of token counts from Open AI: https://platform.openai.com/tokenizer
Regarding token usage, specifically why does 1 question use so many tokens:
Creatio AI performs additional actions beyond the messages generated by the user in the chat. Let me explain with an example:
If you ask Creatio AI to create a draft of an email, Creatio AI performs several steps:
- Search for available skills.
- Read the skill prompt and available actions.
- Read data from the page.
- Check system settings to confirm that the message can be sent.
- Generate a draft message and display it in the chat (the only step visible to the user).
- Perform additional actions, such as opening a card, etc.
Steps 1–4 are not visible in the user interface, but they also consume tokens, which explains why the token usage may appear higher than expected.
Learn more: https://www.creatio.com/products/pricing
Best regards,
Ivan
Hi Ivan
Thanks for your reply. I have looked at the documentation and aware of this information. I was seeing if other users have experiences of your AI implementation. Real world usage is often different, and as our testing showed such different token usage, I was interested to capture this feedback.
Hi Mark,
Same as you: found massive usage tokens consumption for little use.
Unless OpenAI and Azure AI Integration price lowers, for now, use of AI is on hold, until we can plug to other LLMs [which is planned in later subversions of Creatio 8.3 I believe ], some of them being known to be much less expensive then open ai 4o .
Hi Damien,
Thanks for taking the time to reply. This is what I was worried about, as the promised ability to select your own LLM means we are locked into what I hope is the highest tier.
I do not want to tell our team that the AI function is going to be provided and by day 4 of the month, all credits are used up and we would be looking at a very large cost. However, if the benefits are genuine then it is a case of matching to the cost value achieved.
My worry was that the cost of the tokens being so high, makes the value hard to justify.
Same experience for me as what Damien shared. It's a great marketing tool, but real world use doesn't seem to add much benefit IMO, especially compared to the high token usage for even what seems to be simple requests.