Federated　Learning　(FL)　suffers　from　low　convergence　and　significant　accuracy　loss　due　to　local　biases　caused　by　non-Independent　and　Identically　Distributed　(non-IID)　data.　To　enhance　the　non-IID　FL　performance,　a　straightforward　idea　is　to　leverage　the　Generative　Adversarial　Network　(GAN)　to　mitigate　local　biases　using　synthesized　samples.　Unfortunately,　existing　GAN-based　solutions　have　inherent　limitations,　which　do　not　support　non-IID　data　and　even　compromise　user　privacy.　To　tackle　the　above　issues,　we　propose　a　GAN-based　unbiased　FL　scheme,　called　FlGan,　to　mitigate　local　biases　using　synthesized　samples　generated　by　GAN　while　preserving　user-level　privacy　in　the　FL　setting.　Specifically,　FlGan　first　presents　a　federated　GAN　algorithm　using　the　divide-and-conquer　strategy　that　eliminates　the　problem　of　model　collapse　in　non-IID　settings.　To　guarantee　user-level　privacy,　FlGan　then　exploits　Fully　Homomorphic　Encryption　(FHE)　to　design　the　privacy-preserving　GAN　augmentation　method　for　the　unbiased　FL.　Extensive　experiments　show　that　FlGan　achieves　unbiased　FL　with　$10\%-60\%$10%-60%　accuracy　improvement　compared　with　two　state-of-the-art　FL　baselines　(i.e.,　FedAvg　and　FedSGD)　trained　under　different　non-IID　settings.　The　FHE-based　privacy　guarantees　only　cost　about　0.53%　of　the　total　overhead　in　FlGan.