A Travelocity search by proximity to address (in which I entered Newark Penn) revealed only a few rental car offices in the Newark city area. The closest is a Budget Car Rental with reasonable rates ($25-$35 per day for random test dates selected over the next month) and lower taxes than the airport at 474 Main St, Orange, NJ 07050, 4.61 miles away from Newark Penn Station (Budget
location code RA3). Their terms and conditions as viewed through Travelocity (strangely, I couldn't pull these location-specific terms and conditions up at budget.com) specify the following:
PICKUP SERVICE WILL BE CONFIRMED WITH A 24 HOUR ADVANCE NOTICE IF CUSTOMER IS WITHIN 5 MILES OF THE LOCATION, IF LESS THAN 24 HOURS NOTICE AND PICK UP IS FARTHER THAN 5 MILES - THE SERVICE MAY NOT BE AVAILABLE -IN EITHER CASE THE CUSTOMER MUST CONTACT THE LOCAL OFFICE DIRECT TO ARRANGE. . NO SHUTTLE OR DELIVERY SERVICE.
Their number (to set this up) is 973-674-0170.
You may wish to also confirm they can drop you off after you return the vehicle. I don't know the Newark area, but even if Orange is a ritzy neighborhood, it's not good to get stranded 5 miles from your destination! (A cab is, of course, an option, and even with cab fare, this rate and the associated taxes are low enough that you'd still be significant money ahead even if you had to drop $20 on a cab over renting at the airport.)
Also, here are their operating hours (you mentioned coming in on Sunday; note they close at 3pm):
Hours of OperationSunday: 09:00AM - 03:00PM
Monday: 07:30AM - 06:00PM
Tuesday: 07:30AM - 06:00PM
Wednesday: 07:30AM - 06:00PM
Thursday: 07:30AM - 06:00PM
Friday: 07:30AM - 06:00PM
Saturday: 08:00AM - 03:00PM
There is also an Avis a little over 6 miles away, but their rates are MUCH higher (over $100 with taxes!), and they have the same 5-mile pickup radius (meaning that, unless you were able to make special arrangements, they would not be able to pick you up).