Colloidal assembly at fluid interfaces has a great potential for the bottom-up fabrication of novel structured materials. However, challenges remain in realizing controllable and tunable assembly of particles into diverse structures. Herein, we report the capillary assembly of magnetic ellipsoidal Janus particles at a fluid-fluid interface. Depending on their tilt angle, i.e. the angle the particle main axis forms with the fluid interface, these particles deform the interface and generate capillary dipoles or hexapoles. Driven by capillary interactions, multiple particles thus assemble into chain-, hexagonal lattice- and ring-like structures, which can be actively controlled by applying an external magnetic field. We predict a field-strength phase diagram in which various structures are present as stable states. Owing to the diversity, controllability, and tunability of assembled structures, magnetic ellipsoidal Janus particles at fluid interfaces could therefore serve as versatile building blocks for novel materials.