Police say that he asked if he could go inside the school to use the bathroom, but staff refused and he wandered off.
He was then picked up by police and told them that he was going to talk to the CIA, which is based a mile away.
Investigators say he was not armed at the time and made no threats. Officers who detained him then searched his car and found two firearms.
Earlier he had shown up at the agency’s headquarters and told guards, “I’m here and I have a gun,” a law enforcement source told CNN.
Sandow was charged with felony possession of a firearm on school property.
He appeared in court by video link on Thursday but did not enter a plea. A judge ordered him to remain in detention and his next court appearance is set for 24 July.
“There are very likely mental health issues involved,” said Chris Carter, a defence attorney who represented Mr Sandow at the hearing and asked for him to be released.
The preschool said that police would increase patrols around the school following the incident.
“We’re really grateful that the Fairfax County police responded quickly, and this wasn’t anything more than a stressful situation,” Christine Ryu-Naya, a member of Dolley Madison Preschool’s board of directors, said