Ellis is an AI notetaker for in-person meetings. Record your meeting, get a clean transcript with each speaker identified, then ask anything — what was decided, what you missed, how it went. No laptop. No extra hardware. Just your iPhone (or Apple Watch).
Replies
Best
The noisy-room question has a nastier cousin: true overlap, two people talking at the same instant. Diarization picks one speaker per frame, so the quieter voice doesn't get mislabeled, its words just vanish, and nothing in the transcript tells you a sentence went missing. That's harder to catch than plain noise because the output still looks clean. When people talk over each other in a fast brainstorm, does Ellis mark the overlap or just pick a winner?
@dipankar_sarkar ooh this is a great question. Short answer: Ellis just picks a winner. Diarization returns one speaker + text per utterance.
I'm really curious though, for you is this a problem worth solving using tech or meeting cadence? in other words, does this frequently come up?
Report
💡 Bright idea
For me it comes up in fast 3+ person brainstorms and standups, almost never in 1:1s or sales calls where people take turns. I wouldn't chase true source separation, that's a research problem you don't want to own. The cheap win is honesty: when AssemblyAI hands back a low-confidence or overlapping stretch, drop a small 'crosstalk here' marker instead of a clean line, so I know to trust my own memory for that bit. A confidently wrong transcript is worse than one that admits a gap.
@dipankar_sarkar this is great idea. Just to have better understanding for the situation here, can you give me an example of when and how this matters? It is an attribution problem then when you want to give credit, or is it when your own voice get's overlapped? Genuinely curious. Thanks!
Report
The in-person angle is what sets this apart, but recording therapy or doctor visits is also where the privacy question gets sharp. Does the audio and the diarization run on-device, or does the recording get uploaded to a server to transcribe and match against my saved voice profile? And since everyone else in the room never installed the app, is anything about their voice retained, or is it all local to my phone?
@hi_i_am_mimo Short answer: server. Nothing about other speakers' voices is retained.
I see you have an interest in local-first apps and privacy. Curious, would this be a non-starter for you? Thanks!
Report
Honestly, server-side is not a non-starter for me on its own — what decides it is the boundary. Given nothing about other speakers is retained, the two things I would want to verify: is my own audio deleted once the transcript exists, or kept for model training, and is the transcript encrypted at rest so only I can read it? A clear processed-then-discarded policy I can actually check is workable even without full local-first.
In-person is the right wedge, every AI notetaker assumes a Zoom link exists. How do you handle speaker attribution in a noisy room without everyone wearing a mic? That's the failure mode that killed my voice-memo system for coffee meetings.
@chielephant thanks for the question. A few things:
During onboarding you can create a voice profile that gets saved to your account (ie. speaker embeddings via Pyannote)
After each recording it uses AssemblyAI's model for diarization and transcript
UI picker to assign yourself and others to the transcript with best match suggestion
Report
Honestly it's rare by volume but it clusters right where it hurts, the fast back-and-forth when a decision actually gets made, so it feels worse than the raw percentage suggests. I wouldn't chase overlapped-speech separation, it's expensive and still flaky. I'd just surface it: flag the low-confidence diarization spans so I know which two or three lines to re-listen to, instead of trusting a transcript that looks clean. The silent winner-pick is the part that bit me, not the noise.
@robert_dimla Can you tell me more what you mean with overly analyzed? Appreciate your time.
Report
From my perspective, I don't think the technology is the issue. It's more about privacy and consent since this is designed for IRL conversations, how do you encourage users to make sure everyone is feeling comfortable being recorded ?
the "one repository across your whole life" framing is the part that clicks for me, most notetakers box you into work context only. Apple Watch as the recording device instead of needing a phone on the table is also a nice touch for stuff like coffee meetups where pulling out a phone feels weird
no problem, good luck with the launch. curious how the Watch mic holds up in a loud cafe vs a quiet conference room, that seems like the edge case that'd make or break the "just wear it" pitch
good to know it held up against actual street noise, that's a tougher test than most demo videos show. will give it a shot next time I'm in a meeting, curious how it does with crosstalk when two people answer at once
Replies
The noisy-room question has a nastier cousin: true overlap, two people talking at the same instant. Diarization picks one speaker per frame, so the quieter voice doesn't get mislabeled, its words just vanish, and nothing in the transcript tells you a sentence went missing. That's harder to catch than plain noise because the output still looks clean. When people talk over each other in a fast brainstorm, does Ellis mark the overlap or just pick a winner?
@dipankar_sarkar ooh this is a great question. Short answer: Ellis just picks a winner. Diarization returns one speaker + text per utterance.
I'm really curious though, for you is this a problem worth solving using tech or meeting cadence? in other words, does this frequently come up?
For me it comes up in fast 3+ person brainstorms and standups, almost never in 1:1s or sales calls where people take turns. I wouldn't chase true source separation, that's a research problem you don't want to own. The cheap win is honesty: when AssemblyAI hands back a low-confidence or overlapping stretch, drop a small 'crosstalk here' marker instead of a clean line, so I know to trust my own memory for that bit. A confidently wrong transcript is worse than one that admits a gap.
@dipankar_sarkar this is great idea. Just to have better understanding for the situation here, can you give me an example of when and how this matters? It is an attribution problem then when you want to give credit, or is it when your own voice get's overlapped? Genuinely curious. Thanks!
The in-person angle is what sets this apart, but recording therapy or doctor visits is also where the privacy question gets sharp. Does the audio and the diarization run on-device, or does the recording get uploaded to a server to transcribe and match against my saved voice profile? And since everyone else in the room never installed the app, is anything about their voice retained, or is it all local to my phone?
@hi_i_am_mimo Short answer: server. Nothing about other speakers' voices is retained.
I see you have an interest in local-first apps and privacy. Curious, would this be a non-starter for you? Thanks!
Honestly, server-side is not a non-starter for me on its own — what decides it is the boundary. Given nothing about other speakers is retained, the two things I would want to verify: is my own audio deleted once the transcript exists, or kept for model training, and is the transcript encrypted at rest so only I can read it? A clear processed-then-discarded policy I can actually check is workable even without full local-first.
@hi_i_am_mimo appreciate the followup!
In-person is the right wedge, every AI notetaker assumes a Zoom link exists. How do you handle speaker attribution in a noisy room without everyone wearing a mic? That's the failure mode that killed my voice-memo system for coffee meetings.
@chielephant oh nice! you built a voice-memo tool yourself?
I'm experimenting with the following setup:
During onboarding users create a voice profile that gets saved to their account (ie. speaker embeddings via Pyannote)
After each recording it uses AssemblyAI's model for diarization and transcript
UI picker to assign yourself and others to the transcript with best match suggestion
Everyone built for Zoom and forgot rooms exist. How does it handle four people around one table with a single phone mic?
@chielephant thanks for the question. A few things:
During onboarding you can create a voice profile that gets saved to your account (ie. speaker embeddings via Pyannote)
After each recording it uses AssemblyAI's model for diarization and transcript
UI picker to assign yourself and others to the transcript with best match suggestion
Honestly it's rare by volume but it clusters right where it hurts, the fast back-and-forth when a decision actually gets made, so it feels worse than the raw percentage suggests. I wouldn't chase overlapped-speech separation, it's expensive and still flaky. I'd just surface it: flag the low-confidence diarization spans so I know which two or three lines to re-listen to, instead of trusting a transcript that looks clean. The silent winner-pick is the part that bit me, not the noise.
@dipankar_sarkar thanks!
How do you balance giving users AI coaching feedback without making conversations feel overly analyzed?
@robert_dimla Can you tell me more what you mean with overly analyzed? Appreciate your time.
From my perspective, I don't think the technology is the issue. It's more about privacy and consent since this is designed for IRL conversations, how do you encourage users to make sure everyone is feeling comfortable being recorded ?
@reda_roqai_chaoui at the moment using messaging. There are some other ideas to make it even more explicit, such as a consent toggle. Thoughts?
the "one repository across your whole life" framing is the part that clicks for me, most notetakers box you into work context only. Apple Watch as the recording device instead of needing a phone on the table is also a nice touch for stuff like coffee meetups where pulling out a phone feels weird
@omri_ben_shoham1 thanks
no problem, good luck with the launch. curious how the Watch mic holds up in a loud cafe vs a quiet conference room, that seems like the edge case that'd make or break the "just wear it" pitch
@omri_ben_shoham1 Thanks.
Tested today in a cafe with a loud training riding by, and it was good. Haven't noticed yet, where it fails, but I'll continue to monitor it.
Give a try, if you have one. Would love to hear what you think
good to know it held up against actual street noise, that's a tougher test than most demo videos show. will give it a shot next time I'm in a meeting, curious how it does with crosstalk when two people answer at once