TOKYO - A former Japanese justice minister and his wife who are at the center of an election scandal have decided to quit the ruling Liberal Democratic Party led by Prime Minister Shinzo Abe, party sources said Tuesday.