Being around other women my husband slept with really bothers me.?

Within a month I have to go to an event with my husband where several women that he slept with years ago will be at. It truly bothers me!!! I never slept around as a teen and young adult, but he did. We have a very close emotional and very active physical relationship. I can't think of a day that goes my that my husband doesn't tell me loves me and that I am beautiful. Despite our busy lives and many children we still have a lot of sex, are adventuresome with watching videos, sexy lingerie and occassional trips to the "dirty store"...we typically have sex 3-4 times a week. I can't ever think of a time that we had sex whether it was dirty sex or love making that he didn't tell me he loved me during the act. Why then does it still bother me to be around women he slept with like 20 years ago??? I have a huge fear of him standing beside me,but thinking about being with them! Help!!!