The Cost Behind Every Prompt
Through any AI model, whenever you send a prompt into the system, it is broken down into chunks called tokens. A token can represent a word, half a word, a character, or a short phrase. Before the system can even think, it “tokenizes” your inputs to identify patterns and any unfamiliar information. Every single token that is generated, whether it is an answer to your response or question, costs computing power. AI companies pay for the number of tokens used and the extremely expensive computing power.
The increased need for compute capacity is why AI has become a financial burden. AI requires massive amounts of computing power. Training models means running data sets through the model repeatedly across many tailored chips (GPU’s). Second, once a model is trained, it is made available to users, meaning the system must run around the clock, using energy just to process tokens in real time. Finally, the continuous tech arms race means that companies need to continuously create newer, faster, and more expensive chips to stay in the race.
As the price per token falls, it does not mean AI is getting cheaper. AI has become more abundant, so people have started to use it more. This is known as the Jevons paradox. When something gets more efficient, people start to use it more. Cheaper tokens have led to “longer conversations”. These longer conversations use many more tokens to complete one task. As the price per token falls, the total spending keeps climbing.
Why Token Costs Matter More for Businesses Than Consumers
There is a distinct split between consumers and enterprises in who feels the cost. For individual users, adoption is unlikely to be limited by token prices. On average, someone chatting with an AI system on a normal day will ask a few questions that will take up very few tokens. Providers of the AI system can absorb that cost easily by adding ads or a monthly subscription. The main effect is that later providers have to create usage caps or fewer daily messages.
Everything changes once AI is embedded into the workflow instead of being used by a consumer. For example, a company using AI to do background checks on individuals. That takes many steps, including finding data on the person, checking documents, and putting all the information together. This takes many tokens and could require the model to make many tries to find everything it needs. This is only one customer. These AI models are being used for thousands of customers all at once, racking up token usage. This increase in token usage calls for more infrastructure and power, which creates a high cost. The real question for businesses is “does the value per task exceed token cost used on that task?” The cost becomes the deciding factor for companies. They can weigh the token usage to help make the decision.
Token costs alone won’t shape consumers’ or companies’ decision-making. The future of AI costs is dependent on how providers move forward with pricing strategy. Do they stick with a flat subscription that underpays or move towards how many tokens a consumer uses? As AI continues to grow, the amount of tokens being used will increase even more, and companies will have to decide how pricing might change.
