Why did a translation take two seconds?
For most of this year, tapping Translate in baba took about two seconds before the Hebrew finished arriving. We assumed our own code was the problem, so we measured a full week of real translations before changing anything.
It wasn't our code. The first word arrived about 1.5 seconds after the request reached our server, and roughly 1.2 seconds of that was the language model producing its first word. Our own work before the model, checking who you are and saving the translation, took about three tenths of a second.
When almost all of the wait is the model thinking, there is only one way to make it shorter: start earlier.
What does baba do while you are still typing?
baba now begins translating before you tap. Two moments trigger it:
- When you paste. Pasted text is finished by definition, so the translation starts immediately.
- When you stop typing. After a short pause, baba checks whether your message looks complete, then starts.
The work happens quietly on our server. Nothing appears on screen, nothing is saved to your history, and nothing counts against your usage. When you tap Translate, the finished translation is already waiting, and the whole sentence appears at once instead of word by word.
If you edit the text after that, the earlier attempt is thrown away and baba translates what you actually wrote. If you tap before the early attempt is done, baba runs a normal translation at the same time and shows whichever produces text first.
| What you do | What baba does | What you see |
|---|---|---|
| Paste a message | Starts translating right away | The full translation on tap |
| Type, then pause | Checks if you're done, then starts | The full translation on tap |
| Type half a sentence and stop | Waits, because you are mid-thought | Nothing until you finish |
| Edit after pasting | Drops the early attempt | A fresh translation of your edit |
How does baba know you finished typing?
This is the interesting part. A pause is not the same as a finished message. People stop mid-sentence to think, to switch apps, or to find a word. Starting a translation every time someone hesitates would waste a lot of work and still leave them waiting.
We use TypeSafe Jev for this. It answers one question about your text and returns a probability rather than a guess: is this message finished, or is the person still writing it? Below a threshold, baba waits.
The simple rules we tried first were not good enough. Punctuation does not settle it, because "I'll be there in ten minutes" usually has none. Length does not settle it either, because a single word is often a finished lookup. On a test set of complete and half-typed messages, the model got 40 of 42 right, and both misses were complete sentences it judged unfinished, which only costs a head start.
The check runs while you are still on the screen, so it never delays anything you see.
