Fortunately, I downloaded all Amtrak timetables last fall and early this year after they were updated.
In checking the Empire Service timetables, it's clear that the westbound 'through' trains have 10-20 minute scheduled layovers. I've been inside the station at ALB between 448/48 & 449/49 and literally saw 'hordes' of people coming off the trains from NYP. I'd surmise that NYP-ALB is probably one of the largest passenger city-pairs in the east outside of the electrified territories.
Eastbound is another story. Those dwell times go from 36 minutes for the Maple Leaf down to 20 minutes for the Ethan Allen Express Mon-Sat and a full 30 minutes on Sundays probably the busiest day of the week on that train. The added dwell time is likely there as the named trains have gone 400 or more miles vs 300 to/from Buffalo and likely have lost time enroute. The extra time for the Maple Leaf is likely there due to the time variability of getting through customs.
If one studies the timetables and compare times between stations, typically there's a few minutes padding between the last two stations on the route (or maybe last three) vs the other direction. Extra station dwell time may be extra padding or it could be there to handle larger crowds on some days or seasons. And if one of the through trains (even the Lakeshore Ltd) gets there early from the west, there's even longer dwell time because they cannot leave before scheduled time unless they are 'discharge of passengers only' stations and noted they may leave early.
And amid all that, part of creating schedules has to plan on meets or overtakes at specific locations along the way...not only between passenger trains but with freights as well. So there may seem to be areas of slower running or longer station dwell times to stay in their 'slot'. Back before ALB-SDY had the 2nd track restored a couple years ago, dwell times at ALB were likely longer account opposing traffic on single track. Throw in a late-running opposing movement that is already in that single track, westbound trains from ALB would have to wait it out before they could leave the station.
In short, creating a schedule is more like a 'work of art' such that it can usually be 'kept' and in case of delays, much or all of the lost time can be made up through padding. As a passenger aboard trains that are 'killing time' at a station after the work is completed, it's sometimes frustrating that we have to 'wait time' before leaving. Only railroaders and railfans know why the extra wait. The general public doesn't have a clue.