Continuing with the discussion of archetypes from the Hero's Journey ...
This week, our archetype is the Shadow. This is typically, but not always, the villain (not all villains are really Shadows, while not all Shadows are villains). The shadow represents the darkness within the hero, personified into another character, and because the Shadow is largely made up of the "bad" parts of the hero, he's a worthy opponent for the hero and someone it's very difficult for the hero to beat, which means maximum conflict. Facing the worst aspects of himself can ultimately bring out the best in the hero.
I suppose you could think of this archetype as the inverse of the Mentor. If the Mentor is who the hero could become if he fulfills his potential, the Shadow is who he could become if he fails and gives into the worst parts of himself. Voldemort is set up as a Shadow of Harry Potter. Both have similar childhoods, with their families gone and them being unaware of their magical powers. But they react in very different ways. Voldemort sets out to destroy the people he sees as inferior, to make sure that he's never under anyone else's control, while Harry tries to create a community and forms a new kind of family to replace the birth family he lost. Part of the tragedy of the story is the fact that Voldemort could have taken Harry's approach and he'd have had a much happier life (as would everyone else, subsequently).
Quite often, the hero and villain are after the same goal, or at least different sides of the same goal (the detective wants the truth, while the murderer wants to hide the truth). A Shadow villain will be using the traits the hero won't let himself tap into, which could give him an advantage. Indiana Jones and Belloq in Raiders of the Lost Ark were both after the Ark and for a lot of the same reasons. Both had a strong intellectual curiosity and reverence for artifacts from the past. But Belloq let the artifact itself become an obsession, he was willing to sell out to the Nazis to get the support to find it, and he wanted to use its power for himself. Indy had the potential to become obsessed, but he managed to focus on the idea of keeping the Ark safe, and he had more of a struggle because he was more or less on his own instead of supported by the entire Nazi military machine. Because he didn't want the power for himself, he was spared.
The Shadow can also be an institution instead of an individual. We didn't have a single, overall villain in the Firefly universe, just the faceless bureaucracy of the Alliance. But there were times when the way Mal tried to run his ship came dangerously close to being just like the Alliance he was trying to avoid. He thought he knew best and wanted everyone to just do as he said without asking questions. He did usually cave on issues where he was in the wrong because he was ultimately a good guy, but the Alliance represented a lot of Mal's negative qualities that often threatened to tear his crew apart.
This archetype is one that can fit with all the other archetypes at various points in the story, or in various kinds of stories. There can be a dark Mentor who is a Shadow figure -- often seen in tragedies where the Hero is led down the wrong path. The Shadow may sometimes play Threshold Guardian. Shadows are quite often Shapeshifters, as seen with the femme fatales who set up the hero to take the fall or in more literal shapeshifting, such as seen in vampire or werewolf stories. The Hero himself can even be a Shadow, in moments where he's acting on his worst impulses or in situations where he has a split personality. This becomes quite literal with stories like Dr. Jekyll and Mr. Hyde, or with Angel and Angelus in the Buffy universe.
While the Shadow is usually an enemy, this character can sometimes be more of an antagonist, someone who is on the hero's side and even with a common goal, but perhaps with a different plan for getting it, so that he ends up hindering the hero. We may see that in quest stories, where a large party starts out, then ends up splitting because they have different ideas of what to do. The fates of the various factions demonstrate the merits of the qualities they represent.
In my Star Wars examples, Darth Vader is very much a Shadow of Luke Skywalker, given that he's Luke's father. In the original movies, we actually don't see a lot of Luke's negative traits in Darth Vader. Luke is rash and impulsive, and he's more emotionally involved in the fight than might be considered good. He takes it personally instead of looking at the big picture. The Darth Vader we see in those movies is calm, cold and rational, though his pursuit of his son does get into the taking it personally while ignoring the big picture realm. But in the prequels we see that Anakin Skywalker's downfall comes mostly because of those same negative traits we see in Luke. Luke very much has the potential to become like Darth Vader if he gives in to his worst impulses. Meanwhile, we see in the prequels that Anakin is pulled between Mentor Obi-Wan and Shadow Palpatine. Palpatine shares and encourages Anakin's sense that he's somehow special and doesn't have to follow the same rules as everyone else.
Psychologically, the Shadow represents the psychoses, hidden fears and bad habits that can destroy us. While the Threshold Guardian represents that aspects of a person that hold him back and keep him from being successful, the Shadow goes a step further and represents the aspects that can bring about his destruction. By defeating the monsters within in the form of the external monster, the hero, in a sense, heals himself.
The thing to remember about these Shadow characters is that they are characters. They shouldn't just be a bunch of evil traits. The danger of the Shadow is that what he represents can be very appealing, charismatic and attractive. He may offer the easy way out or even sound like he's got a better, more rational plan. Another characterization danger is that the Shadow will be more compelling than the Hero. To avoid that, look at those dark traits in the Shadow and find ways to use them in the Hero to give him some depth and shading. A Hero can make bad decisions and act for the wrong reasons without being evil, and having to figure out where to draw the line makes for a more interesting character and story.
I'm coming to the end of this series, just one more archetype and then a post to tie it all together. So now's the time to raise any questions about other writing topics you'd like me to address.