server multi client

[GiulioZ94]

Is there a way to use one istance of AudioToTextRecorder in a websocket server to handle multiple client at same time?

[KoljaB]

Currently there is no option to parallelize the execution of transcription or voice activity detection. I don't think the underlying libraries (esp faster-whisper) support this, so I think you'd have to write your own wrapper around RealtimeSTT prob using multiprocessing.

[GiulioZ94]

I believe it would be fantastic to have a faster whisper server, along with integrating Realtime STT capabilities with the server. Similarly, implementing Realtime TTS using a Coqui TTS server and integrating it.

[KoljaB]

Managing the load for such multiple client scenarios will be challenging. You can assign different GPUs to faster-whisper to get some degree of parallelism. Stays questionable tho because you'll end up in a 'one GPU per user' ratio, at least for a worst case load scenario.

I'm only developing RealtimeSTT and RealtimeTTS as side projects for my Linguflex assistant btw, so that limits my focus a bit. Currently I don't plan any bigger developments around the infrastructure libs.