With a story in first person, you are intending the reader to become much more attached to the main character. Since the reader sees what that character sees and feels what that character feels, the reader will have an emotional investment in that character. Third person does not have this close tie; a reader can become emotionally invested but it will not be as strong as it will be in first person.
Contrarily, you cannot have multiple point characters when you use first person without extremely clever craftsmanship. You could not subtly move from one perspective to another, and even risk confusing the reader when you do so, which is one of the most egregious crimes any writer can commit. First person stories cannot have the main character die except at the very end, whereas third person stories may be able to use the possible imminent death of any given character as a means for suspense.
Generally, it's a case of emotional attachment vs. utility. It is plain easier to write in the third person, but if you can pull off first person and your story is a good fit for it, it can be a very powerful tool.
Writing in the first person is harder.
Keep in mind that in the first person writing, the main character is also the narrator. He shouldn't be all knowing. It is easier to put down your (character's) thoughts and you can also deceive and lie! (That I like the most. :) )
We also cannot see onto other characters minds, so we can't know how they really feel about our narrator.
Other answers address the differences between first and third person narration but ignore the last type of narration available to a writer - second person narration. Telling a story by dictating to the reader what he or she (the reader) is doing, can create an uncanny, mysterious, eerie and hypnotic effect on the atmosphere of the narrative.
Of course, this is a difficult effect to achieve and if done poorly it can make a story difficult to read.
See Carlos Fuentes' short story Aura for a great example of this type of narration.
First person has the unfortunate side-effect of being narrated in an extreme sort of way. While it's true that the reader has more emotional investment (if done well) in the character, it can also be true that first person can potentially pull the reader out of the story over and over again with side comments that have little to do with what is currently taking place. That's not always bad, but if you have such interruptions (the thoughts of the narrator as he's writing rather than as he's in the story), then you HAVE to make up for it with an engaging writing style. Which is one of the greatest difficulties with first person. If you have such an engaging style, though, then by all means - go for it! If you slip-up, though, it will be much more noticeable and annoying to the reader. You can't just "tell" - you have to tell as your character, who ought to be interesting and fun to listen to.
Genres that especially employ first-person today tend to be more comedic or memoir-based. If the reader goes in with a good laugh, or with the understanding that this is more like someone's diary, then they're willing to deal with occasional stints away from the immediate story when the narrator wants their attention.
There are actually two types of third-person to consider:
There are actually shades of depth to all of these, as well, from deep character penetration to more action-based descriptions, which an author can learn to employ throughout the course of the book, for whichever scene needs them.
I actually disagree about as heartily with the marked answer to this question as it is possible to. First person does not, in fact, make you emotionally closer to the narrator. A lesson I learned the hard way when I wrote an entire novel in the first person only to have the chief criticism come back at me that the main character seemed whiny and self-obsessed; when I was looking for cool and distant but ultimately virtuous and heroic.
It was at that point that the room (for the criticism was delivered in an eight-person writer's circle and the one thing seven people could agree upon was that my heroine was a whiner) dissected the uses of first person perspective. We discovered the following things:
For these reasons it tends to be the case that reading a novel in third person limited becomes a more intimate experience for readers in which they can more easily become attached to the characters and hence feel more positively about them.
In first person the reader is encouraged to form an opinion about the narrator as a very distinct and separate person from them. In a pulp thriller the side-effects of this are minimal because people really mostly care about what is happening to the hero as opposed to being encouraged to care about the hero themselves for purely emotional reasons.
In fact if a viewpoint character in first person keeps referencing how they feel it tends to come across as whining. Thus it is encouraged to keep first-person novels on a lighter less personal basis. For this reason we have to work out a viewpoint character's motivations, feelings, shortcomings that they are unaware of etc for ourselves. This additional burden on the reader to recontextualise the story subtracting the viewpoint character's personal quirks is a distancing experience absent from third-person limited.
Ever since that writer's circle session I have been very careful about employing first-person perspective and would now only do it if I had a very specific reason to convey an unreliable witness or a rube perspective. Generally speaking heroes of genre do not come across as sympathetically in first-person as third. I would not use the first-person to portray a heroic character because nothing takes the shine off a hero like getting deep inside their internal monologue. In third person limited it is much easier to excuse a hero's shortcomings as "features of their personality". In third person it is easier to root for a deeply flawed hero. First person viewpoint characters tend to seem more flawed no matter how much they are outwardly a paragon of virtue.
Here's the most basic difference.
What kind of story is better suited for each one?
That's entirely up to the writer. Whichever one helps tell the story the best.
In all three cases, it is entirely possible to get closer to the character by showing their internal monologue (thoughts). So it's not necessary to tell a story in first person to develop emotional attachment. Deep POV, or Limited Third Person, provides the reader direct access to that character's thoughts and emotions, as much as if told in first person.
For instance, in Jonathan Maberry's Patient Zero, all chapters told from the main character's point of view are written in first person. All chapters told from another character's point of view are written in limited third person (Deep POV). And it works very well. The reader has access to the thoughts and emotions of each character, but only to one character at a time per chapter.
You see this used in thrillers a lot because the main character can never know enough about the events to keep the story interesting. Often, the main character isn't even able to identify the villain until close to the end, so the writer has to give the reader multiple character points of view.
Are there advantages or disadvantages inherent to each form?
Not objective ones. It all depends on the writer and the story they're telling. I will sometimes write a section in two different points of view to see which one tells the story best.
For example, writing in the first person you are always following a character, while in the third person you can "jump" between story lines.
A writer should avoid jumping from one character to another within a logical unit. In most cases, this means sticking to a single character through a chapter, but could also work at the section or even paragraph level if the writer is skilled enough. This part of your question is more about character point of view (which character is telling the story) than point of view (first, second, third).
You rarely see pieces written in second person, but it's not unheard of: List of Notable Second Person Narratives at WikiPedia
James Wood, in How Fiction Works, said that in a first-person narrative, the narrator is talking to someone. I don’t know if that’s a hard-and-fast rule of fiction, but it’s a powerful idea. Let’s say you’re writing a love story with a first-person male narrator. Is the narrator telling this story to his mother? To his lover’s child? To a male friend? To the woman he started dating on the rebound after he was dumped?