Skip to main content

Processing Queue

This article explains details about the SPE asynchronous requests processing queue, the processing task lifecycle, and its handling states.

When SPE receives an asynchronous API request, a task is created and placed in a queue. The size of the queue is defined by the server.n_task_limit setting in the SPE configuration file, with a default value of 1000 tasks.

Tasks in the queue are then handled according to their priority (see Task prioritization section in the REST API documentation and also the SPE processing priority article). Tasks with identical priority are processed following the FIFO principle.

Tasks are picked up from the queue and processed by workers, which allocate instances of technologies for processing. For more details, see Speech Engine and technologies, instances, worker.

The number of tasks that can be handled simultaneously is defined by server.n_workers for audio files processing and server.n_realtime_workers for real-time stream processing settings in the SPE configuration file. By default, this is set automatically, based on your hardware and software configuration. For more information, refer to the How to configure Speech Engine workers article.

The diagram below demonstrates the queue processing (for the sake of simplicity, technology assignments to workers are not shown):

  • Non-colored pending tasks are waiting in the queue (the task state in the response to GET /pending/{ID} request is "waiting").
  • Colored pending tasks are being handled by processing workers (the task state in the response to GET /pending/{ID} request is "running").
  • SPE is configured with 3 standard and 2 real-time workers, i.e., 5 tasks are handled simultaneously (one task has a higher priority, so it gets handled even though it's placed further in the queue).

SPE asynchronous tasks handling

When processing is completed, the task state changes to "finished," and the final result is available for retrieval for a limited time only. The time limit is defined by the server.finished_task_timeout setting in the SPE configuration file, with a default value of 60 seconds. This limit exists to prevent SPE from retaining results indefinitely, which would consume memory.

The result can be retrieved:

  • Using a GET /pending/{ID} request (which responds with HTTP status code 303, redirecting to /done/{ID}).
  • Using a GET /done/{ID} request.
caution

Depending on the development framework used (and/or its configuration), you may not receive the HTTP status code 303 response because the framework handles the redirect internally.

For example, this occurs in Microsoft .NET, as the HttpWebRequest class has the AllowAutoRedirect property set to true by default. Similarly, the WebAPI fetch() method or the XmlHttpRequest automatically follow redirects by default.

After retrieving the final result via a successful GET /done/{ID} call, or after the time limit has passed, the result is removed from SPE and can no longer be retrieved. Subsequent calls to /pending/{ID} or /done/{ID} will return HTTP status code 404.