Your lungs will hurt at first if you're not already deeply into another aerobic sport. It goes away after a few weeks.
Start off with a few miles per ride. Work your way up slowly, adding a half a mile or a mile each time out. It doesn't sound like much progress, but it will always feel like an achievement. And if you were to do it consistently, in a year you'd be riding over a hundred miles a session. Though you'll eventually find time to be your major constraint.
Avoid hills at first, if you can. You're trying to build aerobic muscle and endurance, and hill-climbing builds anaerobic fibers, which confuses the situation. You can train those later, when you're doing longer rides (10+ miles) and the hills will be a smaller portion.
Get the bike set up correctly for you at a bike shop, if it's convenient. If not, here are some rules of thumb: set the saddle so your leg is straight when you place your heel on the pedal. Set the handlebars just below the saddle (at first; as you get more sporting you'll be dropping the bars more). In no case should your knees come above horizontal or your hips wobble from side to side to keep contact with the pedals. Your back should bend just above the hips, and you should be able to bend your elbows or straighten them out at different positions on the handlebars. Your shoulders should droop when you are riding relaxed; not be jammed up behind your ears.
When you're starting out (or coming back after a long layoff) the saddle is your biggest enemy after your lungs. If the bike has a racing-type saddle (narrow, no padding), consider replacing it with something a little cushier for a while, or covering it with a gel-pad. And wear cycling shorts with padding.
If you have clips or cleats on your shoes, practice applying circular force instead of just mashing on the downstrokes. This gets your hamstrings, hip flexors, and abdominals involved. Also practice clipping out. You'll want it to be a reflex when it's time to panic-stop or control your tumble in a crash. Pay attention to any tendon pain in your knees; it may indicate improper alignment of the cleats.
If you stick with it and start to ride more than an hour a day, try not to buy more than 3 bikes the first year. Some will be irresistible, but most aren't worth what they're charging.
Your $35 bike is nothing to be ashamed of. Put a little oil (not WD-40) on anything that squeaks and it'll be a good ride.