The good news is: only if the soap is not made properly. Soapmaking is a chemical reaction that completely uses up the lye in the creation of soap. Modern techniques allow the calculation of a soap recipe so that excess lye is rarely a problem, not like in the old days when lye extraction was a lot more difficult to control.
Fortunately, all reputable soapmakers know how to test their products to avoid the problem of unused lye in their formula. If you are buying soap in a public place, you are likely safe from lye danger.
Fatty acids (weak acid) + lye (strong base) MERGE together to form a neutralized salt, which is a totally new substance. It is a salt, in fact. For more info than you ever wanted about soap as a salt, check our blog.
Salt + water = soap bubbles which CONNECT to dirt but no chemical change occurs
There is no lye involved in washing. Soap works by emulsifying dirt in a prison of water, as we explore here.