Ladd, this sounds like the doings of the APFS disk format. The Finder has become wildly inaccurate with regard to "true" file (and folder) sizes. You should locate an alternate file browser (I like File Buddy) to get an exact and accurate reading of the files you're concerned about.At the beginning of each new year, I duplicate my Apple Photos library (2019) and rename the copy (2020). I then open 2020 and delete all photos/movies from 2018, leaving the 2019 photos and what few photos I've taken in 2020. I then rebuild the 2020 library by holding down the option/command keys when opening Photos.
To my surprise, the Finder still reports the 2020 library as the same size as the 2019 library, 44GB, even though 2020 has approximately half the number of photos/movies in it.
Is there some what I can compact the 2020 library to reduce the file size shown in the Finder?