The story is about a fisherman's family living near to Shanghai. In the beginning, twin babies are born, a boy and a girl. Their growing-up is briefly sketched, and their friendship with "young master", a boy from rich family about their age.

Based on the classic Chinese play Romance of the Western Chamber, Way Down West follows the love story between a poor scholar and wealthy woman. Unable to marry due to their social status, this changes when a bandit attacks the monastery they are staying at demanding the woman's hand. Desperate, her mother promises her hand to whomever manages to defeat the bandit.