If you have more time, you can visit the famous Matrimandir in Auroville or head to Paradise Beach for more water sports activities.
If you have less time, prioritize the attractions that appeal to you and skip the ones that don't. For example, if you're not interested in museums, you can replace the Pondicherry Museum with a visit to the Botanical Garden instead.