Prior to visiting the Highline and Rainbow City, we started our day at Union Square. The Greenmarket there is a favorite of mine, and there are so many fabulous things to see in the area, and so many great places to shop and eat.
One of my favorite walks in NYC is to start at Union Square, walk uptown through the Greenmarket towards Broadway (detour to 19th @ Bway to visit Fishs Eddy, if you so desire), then back to 18th street past the City Bakery, Books of Wonder, The Cupcake Cafe, Paper Presentation, and fire station 14 (built in 1894), turn downtown at 9th Ave to hit up Chelsea Market, and then eventually make it to the Highline at 10th Ave. Long walk, but one of the best routes I know to get a taste of the NYC I know and love (The only route that takes me through Times Square is one that takes me to Junior's Cheesecake).
If we are in the city long-term, I am SO buying one of these- can you imagine biking around the city and having a little trailer with two seats for kids, groceries, etc... I would also need a townhouse or an apartment with a garage (ha!):
Fire House 14 @ 18th Street