Chapter 9. Simple Spider

I once had the pleasure of building a house with a carpenter who was from Mexico; I learned a great deal in the process. Despite my broken Spanish (I once twisted the language so badly that I told him that I was going home to marry my sister), he was able to impart an incredible amount of wisdom. My observations were, shall we say... less wise. I told him that I was proud of my hammer. It had a sleek and heavy head, and a composite graphite handle that made it easy to swing. The head was waffled so it would grip the nails. He simply chuckled. At the end of the day, he had used his 10-year-old, plain-Jane hammer to drive in four times the nails as I did, with more accuracy and precision. He told me, "No esta el martillo:" it's not the hammer.

Luckily, I build code better than I build houses. In this chapter, I continue to put the five basic principles into action. I create a simple service with a light user interface to drive it. The service is an open source web indexer, primarily used to provide site search behavior for a single web site. It is called Simple Spider and is available at http://www.relevancellc.com/halloway/SimpleSpider.html.

I'll give you an insider's view of the real client requirements that spawned the application in the first place. The requirements were minimal and straightforward, but there was still a lot of work to do to understand the problem space. Notice large functionality areas that could be built for this application or reused from other tools; 'll walk you through the decision-making process that led us to use or discard each one. You'll also see the ways our desire for simplicity and our selection of the right tools led to the first iteration of the Spider. In the next chapter, I extend jPetStore to use the Spider. I do this while focusing on the use of transparency to enable extensibility.

Throughout both these chapters, I constantly apply the principle of focus: do one thing, and do it well. You want to build a simple hammer, one that fits like a glove in the hand of a skilled carpenter. That desire affects everything, from the requirements to the design to the individual lines of code you write. Along the way, I aim a spear at a variety of sacred cows, from over-burdensome frameworks and over-expensive data formats to notions about how technologies like JUnit and HTTPUnit should be used. In the end, you'll have an open source application and a greater appreciation for how good programming in Java can be if you just keep your wits about you.