Despite the crucial role of agriculture in economic development, the impact of investments in this sector remains little known, and information that could inform policy and encourage further investments for rural poverty reduction is scarce. We assess the impact of agricultural investments on labor productivity and poverty reduction in rural areas in Togo by analyses of data collected from 4,543 households in four townships via a Community-Based Monitoring System approach. Through descriptive statistics and econometric modelling of agricultural households, we examine such issues as gender disparities, educational levels, water and sanitation, insurance, and income. We find that public investments in health education and telecommunication infrastructure improve farmers’ productivity and reduce the number of farmers below the poverty line, as do farmers’ investments in inputs. We note an estimated farm-income gap between men and women of 44.1%, and suggest that increased public and private investments in education, health, safe drinking water, and irrigation infrastructure, as well as improvements in the equitable distribution of endowments such as land would equalize opportunities for disadvantaged or excluded groups and reduce the income gap between men and women.