Island Man portrays the life of Jim Hardie who lived and worked on Bear Island in the middle of Penobscot Bay, Maine, from 1910 until 1946. An orphaned child, at the end of the 19th century, Jim was sent from Scotland to Canada, ran away from his foster home on Prince Edward Island, shipped out to sea and worked on sealing vessels in the Antarctic for ten years before arriving in Maine in 1907. Living at the Harbor House on Bear as the caretaker beginning in 1910, Jim and his wife Alice and seven children were the only year-round occupants of the island.

A self-sufficient, self-taught man, with a "gargantuan will," Jim learned how to read and write, and manage and perform all of the tasks needed as caretaker of the island, including accommodating the needs and wishes of the island owners, who were summer visitors. He was a fisherman, a farmer, a carpenter, a builder, and a "captain," as caretakers of islands were called. His only instruments as he navigated the open waters were his eyes, his wristwatch and his compass. Jim played a special role in the lives of the children of the summer family. One of those is the author's father, Wolcott Fuller, who collected the letters that Jim wrote to him, over the course of Jim's life, from his years at Bear until he moved to neighboring Scrag Island in 1946, where he died in 1954.

Excerpts from the letters frame the narrative of the book, accompanied by over fifty sepia-toned photographs of island landscapes and of the Hardies at home and work. In addition, there are five maps of the island, surrounding locations, and Jim's early travels. The originals of seven of Jim's letters are reproduced in an appendix. Together the text and illustrations tell the remarkable story of Jim's hard scramble, independent life of survival and sustainability.