Some fragrances will effect the color of soap. If you are using a fragrance oil that contains vanilla (vanillin), then unfortunately, your soap will turn brown. Vanillin is the organic compound that gives vanilla its flavor. Vanillin discoloration is a 100% natural process that can be slowed by adding chemicals to stabilize the fragrance. This effect is most noticeable in cold process soap, but will occur in all soaps and lotions.
Even then, the reaction and discoloration will still happen over time. This color can range anywhere from a slight yellow all the way to a dark brown (the same color as a dried vanilla pod and real vanilla extract).
Some fragrance oils may contain vanillin as a constituent even if Vanilla itself does not appear in the name which is why we test all of our fragrance oils to determine their behavior in cold process soap.
Here is a Soap Queen blog that explains this further, and also lists our fragrance oil's vanillin content percentages:
If you are working with melt and pour soap, you can use our Vanilla Color Stabilizer in equal parts with your fragrance oil and it will off-set the vanillin turning your soap a dark brown for a time.
Also, if you absolutely do not want your fragrance oil to change the color of your soap, I would suggest trying a water white fragrance. These are fragrance oils that are crystal clear and will not affect the color of your soap at all.