Measuring energy access through binary indicators is insufficient, and in some cases even misleading. In this work, we critically evaluate the World Bank's multi-tier framework (MTF) to measure household electricity access using a household survey in rural Bangladesh. We argue that the MTF addresses multiple objectives, thereby offering less value as a single composite index than as a set of dimensions along which to evaluate different aspects of electricity access. We test the robustness of the framework to alternative specifications as  regards the choice of attributes and tier thresholds. The study shows that access measurement is highy sensitive to changes in parameter values, the application of different algorithms, and data availability. We also discuss the wider implications of applying the  framework to current electricity access intervention programs in Bangladesh, provide feedback on the TF's design, as well as suggest potential improvements for its application in the future.