StepFun
Scale-up intelligence possibilities for everyone
349 followers
Scale-up intelligence possibilities for everyone
349 followers
StepFun is a frontier AI platform building powerful, actionable intelligence with the Step series of models.
This is the 3rd launch from StepFun. View more
Step 5 Preview
Launched this week
Step 5 Preview is StepFun's open-weight MoE for coding, financial research and long-running agents. It has 600B total parameters, 27B active per token and a 1M-token context, with deployment support for vLLM, SGLang and Transformers.
Launch Team
Hi everyone!
StepFun has released the weights for Step 5 Preview. It was already available via API; now the 600B MoE is on HF. Only 27B parameters are active per token, with a 1M-token context window.
Their technical write-up goes into how they train for long-running agent tasks. They went with 92 transformer layers, sparse attention for the huge context, and even made context compaction part of the RL training, so agents learn what to preserve when they run out of room.
In one test, they gave it 24 hours to optimize an H100 GPU kernel. Its best run reached 508 TFLOPS, compared with 493 for Opus 5 in StepFun's test.
They also let it play Pokémon Red for over 3,000 turns and 6 million tokens. Three Gym Badges, and still only about a third of the way through!
FP8, BF16 and NVFP4 weights are available, with vLLM and SGLang deployment guides.
Amazing work, cant wait to try it, I wish you also had a RLCD as well for fast decision making, would that be in the works soon as these models would work great if you had a router for reasoning and decision making