Binary neutron star mergers play an important role in nuclear astrophysics: their gravitational wave and electromagnetic signals carry information about the equation of state of cold matter above nuclear saturation density, and they may be one of the main sources of r-process elements in the Universe. Neutrino-matter interactions during and after merger impact the properties of these electromagnetic signals, and the relative abundances of the produced r-process elements. Existing merger simulations are however limited in their ability to realistically model neutrino transport and neutrino-matter interactions. Here, we perform a comparison of the impact of the use of state-of-the art two-moment or Monte-Carlo transport schemes on the outcome of merger simulations, for a single binary neutron star system with a short-lived neutron star remnant (). We also investigate the use of different reaction rates in the simulations. While the best transport schemes generally agree well on the qualitative impact of neutrinos on the system, differences in the behavior of the high-density regions can significantly impact the collapse time and the properties of the hot tidal arms in this metastable merger remnant. The chosen interaction rates, transport algorithm, as well as recent improvements by Radice et al to the two-moment algorithms can all contribute to changes at the level in the global properties of the merger remnant and outflows. The limitations of previous moment schemes fixed by Radice et al also appear sufficient to explain the large difference that we observed in the production of heavy-lepton neutrinos in a previous comparison of Monte-Carlo and moment schemes in the context of a low mass binary neutron star system.