This is a well-known story, but very moving nonetheless. Here is J. A. Joel's account of the Passover seder he and twenty fellow Union soldiers who were Jewish were able to hold (for both nights) in West Virginia in 1862.
Only 19 years old at the time, he writes about securing permission to take off, and the manner in which they were able to acquire matzos, their delight when it arrived containing haggados and prayer books, and some - but not all - of the necessary food. They could find no horseradish, but used a very bitter weed. In lieu of charoses, they used a brick. Not knowing which part of the lamb to use for the zeroah, they put an entire lamb on the table (and ate it afterward). He describes how some of them got drunk from the cider, which they used for the four cups (or, possibly, more). His letter was printed in the Jewish Messenger April 1866.
As I was preparing this post, I discovered there is even a children's book loosely based on this letter and the event described:
And here is Joseph Joel, apparently he sent this photograph to his friend Rutherford B. Hayes (link):