COMPOZELABS

Voice agents have to be lightweight because voice is data-intensive, so the models under the hood aren't as smart. We got around this by having observer agents looking at the transcript in real time. These observer agents used the latest reasoning models which are really intelligent and we could put really large instructions into those. They would look at the transcript and inject guidance—guidance came in the form of additional user input wrapped in tags indicating 'Hey, this is not actually the user speaking, this is kind of like the system instructions.' Would say stuff like: 'Oh, you need to object more to this part' or 'You need to wrap up the call now.' In this way, we got a more intelligent, better instruction-following voice agent by having this dual system.

Voice mode by itself will never stop talking to the user. We had to give it a custom tool for hanging up—a tool that was essentially the 'end yourself' tool that would make it so the agent could end its own process, hang up the phone, and close the connection.

It could search a user's LinkedIn profile. You just plugged in their LinkedIn and it retrieved everything, scraped the LinkedIn page for everything there was to know. It created structured output—a profile representation that included personality, job, personal details. It would also make up additional stuff if information wasn't available that we wanted. It then used that as part of the system prompt for the voice AI.