RX J1604 is a dipper source, and the data show different kinds of variability. However, a detailed analysis of repeated measurements shows that the results are not affected by dipping events or atmospheric seeing variations. The bright disk of RX J1604 has a very simple axisymmetric structure and is therefore well suited as a benchmark object for accurate photo-polarimetric measurements. We derive accurate radial disk profiles for the intrinsic polarized intensity, Q{phi}(r)/I{star}, and measure different profile peak radii for different bands because of the wavelength dependence of the dust opacity. The disk-integrated polarization is Q{phi}/I{star}=0.92+/-0.04% for the R band and 1.51+/-0.11% for the J band, indicating a red color for the polarized reflectivity of the disk. The intensity of the disk is I{disk}/I{star}=3.9+/-0.5% in the J band, and the fractional polarization is p{phi}=38+/-4% for the J band and 42+/-2% for the H band. The comparison with the IR excess for RX J1604 yields an apparent disk albedo of about {Lambda}{I}~0.16+/-0.08. We also find that previously described shadows seen in the R band data are likely affected by calibration errors. Using dust scattering models for transition disks, We derive approximate J band values for the scattering albedo {omega}~0.5, scattering asymmetry g~0.5, and scattering polarization p_max_~0.7 for the dust. Because p_{phi}_ and {Lambda}{I} depend predominantly on dust scattering parameters and only weakly on the disk geometry, these parameters define tight relations for the dust scattering parameters {omega} and p_max_ and {omega} and g. The positive R to J band color for the polarized reflectivity, (Q{phi}/I{star})_J_~1.64x(Q{phi}/I{star})_R_, is mainly a result of the wavelength dependence of dust parameters because the scattering geometry is expected to be very similar for different colors. This work demonstrates the potential of accurate photo-polarimetric measurements of the circumstellar disk RX J1604 for the determination of dust scattering parameters that strongly constrain the physical properties of the dust.