We present a long-exposure (~10hr), narrowband image of the supernova (SN) remnant Cassiopeia A (Cas A) centered at 1.644{mu}m emission. The passband contains [FeII] 1.644{mu}m and [SiI] 1.645{mu}m lines, and our "deep [FeII]+[SiI] image" provides an unprecedented panoramic view of Cas A, showing both shocked and unshocked SN ejecta, together with shocked circumstellar medium at subarcsecond (~0.7" or 0.012pc) resolution. The diffuse emission from the unshocked SN ejecta has a form of clumps, filaments, and arcs, and their spatial distribution correlates well with that of the Spitzer [SiII] infrared emission, suggesting that the emission is likely due to [SiI] not [FeII] as in shocked material. The structure of the optically invisible western area of Cas A is clearly seen for the first time. The area is filled with many quasi-stationary flocculi (QSFs) and fragments of the disrupted ejecta shell. We identified 309 knots in the deep [FeII]+[SiI] image and classified them into QSFs and fast-moving knots (FMKs). The comparison with previous optical plates indicates that the lifetime of most QSFs is >~60yr. The total H+He mass of QSFs is ~0.23M_{sun}_, implying that the mass fraction of dense clumps in the progenitor's mass ejection immediately prior to the SN explosion is about 4%-6%. FMKs in the deep [FeII]+[SiI] image mostly correspond to S-rich ejecta knots in optical studies, while those outside the southeastern disrupted ejecta shell appear Fe-rich. The mass of the [FeII] line emitting, shocked dense Fe ejecta is ~3x10^-5^M_{sun}_.