You could also continue down the SLR to Yarmouth Jct. where the line crosses the Brunswick Branch. There is a connecting track that would have to be reinstalled for the train to switch to head west on the Brunswick Branch. It would have the advantage of minimizing the amount of the CSX (former PanAm) freight main that would have to be used, basically the same as the current Downeaster.
But this is all a pipe dream anyway, I agree with what @Ticooper says above, the most practical way to do this is via Springfield. But it is fun to speculate.
If I'm remembering correctly, the line from Yarmouth Junction north is what NNEPRA has been discussing as a way to extend the Downeaster up to Auburn, apparently to avoid conflicts on the CSX line. So if they get some money from the federal infrastructure law to do this, that route might become a viable option for service farther north, given that the St. Lawrence & Atlantic is in much better shape north (west) of Danville Junction. But from there down to Yarmouth Junction looks pretty decrepit at the moment.
And yes, if the idea is to have an Amtrak service from Boston to Montreal, a routing via Springfield seems like it could be achieved much more easily, given that only the portion north of St. Albans doesn't currently have Amtrak service. Of course, north of the border it faces the same issues that the Adirondack has. And the trip time would be so long that it would probably only be tolerable as an overnight run.
There is, however, a private operator in Canada, discussed in another thread, who has been talking up the idea of a Boston-Montreal overnight train with a routing via Sherbrooke, Portland and Old Orchard. I find it hard to believe it will ever happen, but it's nice to dream.