While this depends on where you come from (and therefore which dialect of English and which accent you speak with), following standard pronunciations, the ‘th’ sounds are generally the same in those words.
Using the International Phonetic Alphabet, though is written as /ðəʊ/ for both Standard American and Standard British pronunciations. That is written as /ðæt/ for both as well.
The /ð/ represents a voiced dental fricative (the ‘th’ in though and that rather than the ‘th’ sound in thin, which is a voiceless dental fricative represented by /θ/).
If you want to learn more about the International Phonetic Alphabet, you can find a guide here: International Phonetic Alphabet and Phonemic Alphabets