This is cool. But is there a custom layer in front of these models that can slow them down?
Report
Maker
@chilarai yes, there’s a layer, we have a redis queuer in between the API and our GPU workers to help handle the load.
Basically, the API and queuer validate the request, push a job into the queue, and return a job id. When our worker finishes it calls our webhook, we store the result, and you can fetch your image given the job id.
We don’t do any heavy pre or post-processing that would significantly slow the models down. The orchestration layer only adds a few milliseconds compared to calling the models directly.
Report
Maker
Feel free to test using one of the prefunded (25 credits) API keys below. You can test it on your browser: documentation
Swytchcode
This is cool. But is there a custom layer in front of these models that can slow them down?
@chilarai yes, there’s a layer, we have a redis queuer in between the API and our GPU workers to help handle the load.
Basically, the API and queuer validate the request, push a job into the queue, and return a job id. When our worker finishes it calls our webhook, we store the result, and you can fetch your image given the job id.
We don’t do any heavy pre or post-processing that would significantly slow the models down. The orchestration layer only adds a few milliseconds compared to calling the models directly.
Feel free to test using one of the prefunded (25 credits) API keys below. You can test it on your browser: documentation
16bff8f7-5fb3-4fd3-8fd3-a1d9d93624e0
ad809b25-d172-4c3b-8c40-a6894351dc1f
80c6ea1b-3273-4d99-8265-d3ad407cc01a
6c136078-4ef2-4ba8-9173-e0ac9da9c0f0
2a98b55d-9644-434b-8d08-8b9fabd420c3
b649990e-41e1-494e-bcc8-1495a528dff4
219fea7d-3de7-4c99-9f59-c075f5697719
73e60eed-70fa-4a9f-98dc-5eb113d69242
51134ebe-c353-43ff-ac3c-426c3e7d6559
afd6c5f4-5d4f-4a60-a690-c87be0ae350b