The magnetic drivers of campfires seen by the Polarimetric and Helioseismic Imager (PHI) on Solar Orbiter