Passive Design Homes have to meet 7 different sets of criteria; and high performance windows and doors are one of them.
It's all about slowing the conduction of heat and "controlled" air exchange.
1. Super insulate- which works on the same principles as a thermos.
2. Eliminate thermal bridges- no material should have a direct connection with the interior and the exterior.
3. Make it air tight- this provides a "measurable" dimension of the quality of construction.
4. Specify energy or heat recovery ventilation. A house needs to breathe but rather than breathing unknown volumes of air through uncontrolled leaks, a passive home breathes controlled volumes of air by mechanical ventilation that measures the amounts of fresh air throughout the home and exhausts known quantities of stale air from the house.
5. Specify "high performance" windows and doors.
6. Optimize passive-solar and internal heat gains. The orientation of the building and it's various parts will affect energy losses and gains.
7. Model energy gains and losses using the PHPP. All of the elements need to be integrated with one another in a passive house design. This includes wall thickness, R or U-values, thermal bridges, air tightness, ventilation sizing, windows and doors, solar orientation, climate, and energy gains and losses.
If you don't think windows and doors are "extremely" important to a homes comfort level, noise reduction, heating, cooling, interior fading, and aesthetics; THINK AGAIN.