When Paul used the word “Judaism” in Galatians 1:13-14, what was he talking about?

In my book, I argue that the rare word Ἰουδαϊσμός–which is often translated “Judaism” in our Bibles (Gal 1:13-14)–doesn’t mean “Judaism” in the modern sense of a system of religious thought. Rather, it should be understood in a vocational sense:
Paul and the Vocation of Israel: How Paul's Jewish Identity Informs his Apostolic Ministry, with Special Reference to Romans

At times, Paul goes out of his way to reject a particular expression of Jewish vocation, a vocation to which he had once been singularly dedicated. He refers to this vocation using three interrelated terms: Ἰουδαϊσμός (Gal 1:13, 14), ζῆλος / ζηλωτής (Gal 1:14; Phil 3:6; Rom 10:2), and Φαρισαῖος (Phil 3:5). The first two terms are often rendered in English with direct transliterations: “Judaism” and “zeal[ot].” To the modern reader, these English transliterations suggest that Paul is speaking about a general system of religious thought called “Judaism,” which he once “zealously” followed but which he has now rejected. However, as we shall see, these English transliterations do not accurately render the meaning of the Greek terms. Ἰουδαϊσμός and ζῆλος / ζηλωτής, along with Φαρισαῖος, do not refer to the Jewish religion in general. Rather, they refer to a particular expression of Jewish vocation—a vocation which construed Israel’s role in God’s purposes as a call to live as a holy nation in the midst of the other nations, and to preserve that holiness by seeking to protect and remove Jews from the contaminating influence of sinful and unclean Gentiles. (p. 89)

NB In our extant sources prior to or contemporaneous with Paul, the word Ἰουδαϊσμός occurs only in 2 and 4 Maccabees (2 Macc 2:21; 8:1; 14:38 [2x]; 4 Macc. 4.26) and twice in inscriptions.

The details of the argument and further references may be found in chapter 3 of the book (pp. 89-90). The chapter is available from the publisher in electronic format:

Windsor, Lionel J. Paul and the Vocation of Israel: How Paul’s Jewish Identity Informs his Apostolic Ministry, with Special Reference to Romans. BZNW 205. Berlin / Boston: Walter de Gruyter, 2014.