Yes, everything changed since detraining with luggage in each direction, VIA Rail cutting its other frequency, and the extremely high jump in through fares once in Ontario, VIA Rail's attempt to compensate for its poor riderhip.
I last looked at a VIA Rail annual report about 4 or 5 years ago. Taking that route's (with one frequency) annual ridership and dividing by trains per year, it came out to 43 per trip (international + domestic). It was also one of the financially worst performing trains in the entire VIA Rail system, ballpark with Quebec remote trains.
Amtrak website doesn't show state facts sheets for Ontario and Quebec, but from my expereince, every Adirondack load into Montreal (there is no domestic Canadian travel) is in the 121 - 144 range, that number I hear the conductors speak over the radio through my scanner. They are spread through 3 coaches, on each side of the dinette.
It makes sense to me that the schlep-your-luggage-into-the-station drill would drive away business. It might be tolerable for younger adults going on a weekend trip with a backpack and/or a small roller bag. But it's really a hardship for families with children, older folks and those with limited mobility, and anyone with luggage enough for a two- or three-week vacation. (My family's trips typically include two or three of these categories, so I made a note to myself to steer clear of the Maple Leaf after we had to haul ourselves and our stuff off for customs at NFL in 2017.) Even so, I'm surprised the ridership would have fallen so dramatically so quickly.
I did notice when I booked a solo trip a few years back that I could save more than $30 US by booking an Amtrak ticket to NFO and a separate VIA ticket from there to Toronto rather than paying the Amtrak through fare.
The growth of the through traffic on the Adirondack has been impressive and I think was under way for quite a few years before the Maple Leaf customs changes. I can remember some trips in the early '90s when there were only a couple dozen of us crossing into Canada after all the SUNY students detrained at Plattsburgh.