Hello Vignesh Selvasundar
It seems you are finding a tough time organizing your contacts. You can try Agile CRM for contact management. It's a very user-friendly feature just as you are wishing it to be.
The contact management stores all your contacts and their details in a single page view sorting out your chances of confusion while viewing a contact.
Create a detailed profile for each contact with contact information, communication history, social media interests, etc.
Add notes for a specific contact, use customizable tags, segment the contacts as per your convenience.
Agile CRM is a very good bargain for a freelancer as its free for up to 10 users!
I will the link here. Do give it a try!