The 770 first. It would be easily able to push that resolution. You will be wasting its potential on the old monitor. but if you get a higher resolution monitor before you get a gpu you likely won't be able to play games at the higher resolution anyways.
Tl; Dr: 770 before monitor. You might actually like to play your games.
The thing is, if you delay the second purchase too much, it might be a waste either way because it might become obsolete.
The 770 goes well with a 1080p screen. If you buy both in a close enough time frame, that's good. But if you wait a little, you might want to go for 1440p or even 4K, and the part that you already bought will be obsolete.
So your current screen is 1600x900? What is your current GPU? That's the essential to decide which one to replace first.