The Independent Order of Odd Fellows began in 1819 in Baltimore, Maryland. Originally, the Odd Fellows was a type of fraternity lodge that valued charity and benevolence and would visit the sick during the yellow fever epidemic in Baltimore. Due at least in part to the epidemic, burying the dead became a key activity of the Odd Fellows, and their lodges began to purchase land throughout the United States to make into cemeteries. The Odd Fellows expanded to Milford and began a cemetery in the city, which later became most of the Milford Community Cemetery.

Another section of the Milford Community Cemetery started as a cemetery for African Americans during the time of segregation. A brick wall was built between the rest of the cemetery and the African American Section, which was known as the Levi Cemetery and was cared for by “A Society of Colored Men of Milford” known as the “Levi Club.”

When the Milford Odd Fellows Lodge disbanded, they deeded the entire cemetery over to a nonprofit, known as The Odd Fellows Cemetery of Milford, Inc. to care for the cemetery. The nonprofit currently cares for the cemetery, ruled by a Board of Directors on which several Milford city officials sit.