Anyone else struggle with knowing what to say no to?

When I started I thought saying no would be easy. If something didn’t fit the product I’d just leave it out. But as things grew every request started feeling important. Some were from customers some from people on the team and some just seemed like opportunities I didn’t want to miss. How do you know when saying yes is helping the business grow versus just making it harder to focus?