All activity
Tokenflood allows you to 1) figure out how to slash LLM latency by adjusting prompt parameters 2) assess the load curve of LLM providers before going to production with them
TokenfloodFigure out who or what is stealing your LLM latency
Thomas Werkmeisterleft a comment
Hey folks, I just released a new version of tokenflood featuring an all new data viz dashboard and observation mode. Observation mode allows you to track an endpoint's latency over a longer period of time before sending your prod data there. Basically, you can find out at what time during the day everybody starts stealing your LLM latency 😉. TLDR: figure out how to slash LLM latency by...
TokenfloodFigure out who or what is stealing your LLM latency