what bothers me is that Amazon is filtering results in this way in the first place. The purpose of a search engine is to return relevant results, not to try to decide what's "fit" for me to read.

The ranking system is basically telling you what others thought was best, not what Amazon Inc. does personally (corporately). It is aggregate data, probably somewhat mysteriously arrived at via combining a bunch of factors, like Google's PageRank. But aside from that, in this case it sounded to me like the ranking values in some books' records were either being blanked out or ignored, so they wouldn't show because the rank was null as far as the page generating code was concerned. (You'd think something with "zero" rank would simply show at the bottom, but it's possible that it was really "null" or something else the code would consider corrupt, and therefore skip the records entirely.) That such a thing could happen is perhaps disturbing (certainly frustrating), but I don't think this is designed to be a filtering process used as such on purpose, rather a side effect of certain data being erased or ignored.