Launching today
VECTRA by Valley Semiconductor
We built an AI processor that beats Jetson Orin on Qwen 27B.
1 follower
We built an AI processor that beats Jetson Orin on Qwen 27B.
1 follower
VECTRA is a programmable AI processor running Qwen 27B on our FPGA prototype, bit for bit against our 4-bit reference. It reached 7.9 tokens/sec in short-context decode, ahead of a Jetson AGX Orin in our tests. Built for robots and autonomous systems.
Launch Team
AWS Activate CreditsCredits that fuel your startup’s next milestone
Promoted
Hi Product Hunt, I'm Jack, founder of Valley Semiconductor.
Go to valleysemiconductor.com, ask a question, and watch a 27B language model answer from our own processor design running on an FPGA. No GPU involved.
7.9 tokens/sec on Qwen 27B in short-context decode, about 5.5 in live chat
4.2x faster in eight days, 533 ms to 126 ms per token
Faster than Jetson AGX Orin in our short-context tests, compared with 7.4 tokens/sec in MAXN mode and 5.9 tokens/sec in 50W mode
Every output matches our 4-bit reference bit for bit
Qwen 27B, Llama 3.1 8B and OLMoE on one FPGA image
For a 27B model, every token means moving about 13.5 GB of weights through memory. VECTRA is built around that one fact. The ASIC is designed for about 12 tokens/sec at 16 W, and Goodman Technologies has signed a letter of support to evaluate it on their autonomous ground vehicle.
It's one FPGA answering one question at a time, so you may wait a moment, or you can watch a recorded run. If you build robots, drones or autonomous vehicles, tell me what you'd run on it.