We present a statistical study of the relative orientation in the plane of the sky between interstellar magnetic fields and filaments hosting cold clumps. For the first time, we consider both the density of the environment and the density contrast between the filaments and their environment. Moreover, we geometrically distinguish between the clumps and the remaining portions of the filaments. We infer the magnetic field orientations in the filaments and in their environment from the Stokes parameters, assuming optically thin conditions. Thus, we analyse the relative orientations between filaments, embedded clumps, internal and background magnetic fields, depending on their environment and evolutionary stages. We recover the previously observed trend for filaments in low column density environments to be aligned parallel to the background magnetic field; however, we find that this trend is significant only for low-contrast filaments, whereas high-contrast filaments tend to be randomly orientated with respect to the background magnetic field. Filaments in high column density environments do not globally show any preferential orientation, although low-contrast filaments alone tend to have perpendicular relative orientation with respect to the background magnetic field. For a subsample of nearby filaments, for which volume densities can be derived, we find a clear transition in the relative orientation with increasing density, at n_H_~10^3^cm^-3^, changing from mostly parallel to mostly perpendicular in the off-clump portions of filaments and from even to bimodal in clumps. Our results confirm a strong interplay between interstellar magnetic fields and filaments during their formation and evolution.