Police say Wayne Field had on Williams' clothing and World Series ring. Williams was out of town when the incident occurred in November.
The suspect's name matched that on a hospital bracelet found after an earlier break-in at a townhome.
Field had lived in an Uptown homeless shelter and is now in Cook County Jail. He's expected to be transferred to a mental health facility.