Hoffman and Novak's Critique of the Rimm Study

This document is a part of a He Says / She Says set of debates over the TIME cover story "Cyberporn" and the Rimm study upon which it was based. It is has been modified only to add links to related parts of other statements in the ongoing debate. The original version of this text can be found here.

A Detailed Analysis of the Conceptual, Logical, and Methodological Flaws in the Article:

"Marketing Pornography on the Information Superhighway"

July 2, 1995 (version 1.00)

Donna L. Hoffman & Thomas P. Novak
Associate Professors of Management
Co-Directors, Project 2000
Owen Graduate School of Management
Vanderbilt University

Permisson to repost is granted. Copyright (c) 1995 retained by Hoffman and Novak.

In this critique, we provide a detailed analysis of the recent
article "Marketing Pornography on the Information Superhighway..."
(Rimm 1995, Georgetown Law Journal, Volume 83, June, pp 1849-1934)
that was also the subject of a recent Time cover story (Elmer-
DeWitt, July 3, 1995).  For a detailed critique of the Time article, 
see Hoffman & Novak (July 1, 1995, version 1.01).

First, we offer general comments about the study.  We
criticize the study on conceptual, logical, and process grounds, including:
1) misrepresentation, 2) manipulation, 3) lack of objectivity, and 4)
methodological flaws.  Second, we provide a series of detailed examples
that support our general conclusions of these four major difficulties. For
ease of exposition, these specific comments follow the order of the

Our objective with this note is to begin a constructive
and open critique process.  Thus, our note is not meant to be an exhaustive
cataloging of the lapses, discrepancies, inconsistencies, and errors in
this article, only a summary of those we consider to be among the most

We do not debate the existence of pornography in
"cyberspace."  Indeed, pornography exists and is transmitted through many
media, including cable television, books and magazines, video tapes,
private "adult" bulletin boards, the postal mail, computer networks,
interactive multimedia like CDROM, fax, and telephone, to name a few.  What
we dispute are the findings presented in this study concerning its extent
and consumption on what Rimm calls the "information superhighway."  The
critically important national debate over first amendment rights and
restrictions on the Internet and other emerging media requires facts and
informed opinion, not hysteria.

The critique is also important because the Time cover story has
given the Rimm study a credibility it does not deserve.  If it is
difficult for a professional journalist to evaluate the validity of
such research, it is reasonable to assume that many others will
have difficulties, as well. 

The general and specific comments that follow represent our
professional opinion of the critical flaws and errors in the Rimm
study.  Our critique has benefitted from the impassioned
discussions in the WELL Media conference (topic 1029), of this
study and the larger debates of media responsibility and first
amendment rights.   


General comments

*       Misrepresentation 

        *      The study is positioned as "marketing pornography on the
               information superhighway."@  Yet, it deals neither with
               marketing nor the information superhighway, and displays
               a considerable lack of understanding of both areas.

               Out of 148 footnotes in Rimm's manuscript, only one
               (footnote 22)@ cites a reference to a marketing journal. 
               For a study purporting to deal with "marketing
               pornography on the information superhighway," this
               demonstrates a blatant disregard and ignorance of the
               marketing literature.  As "marketing" appears as the
               first word in the title of this manuscript, and the word
               "marketing" appears frequently throughout the manuscript,
               it is particularly disturbing that Rimm does not support
               any of his "marketing" insights with references to the
               marketing literature.

        *      This is a study of descriptions of pornographic images on
               selected adult BBS in the United States.  The author
               finds, not surprisingly, that adult BBSs contain
               pornography.  While the author attempts to generalize
               beyond this domain to the "Information Superhighway,"@ no
               generalization is possible, and the results of this study
               should not be used for this purpose. Unfortunately, the
               juxtaposition of unrelated analyses of adult BBSs and
               Usenet newsgroups may create in the casual reader's mind
               the impression that what is stated about adult BBSs is
               also true of the "Information Highway" as a whole.  We
               caution, in the strongest possible terms, against such
               misinterpretation on the part of the casual reader.   

               Rimm concludes his study by saying (p.1915) "These and
               other findings may assist policymakers and others
               concerned with the future of Cyberspace to make informed
               decisions, with reliable data, about the evolving
               Information Superhighway."@  Unfortunately, this paper
               provides no actionable insights for policymakers about
               the future of Cyberspace, as the results, at the maximum,
               can only be generalized to all adult BBSs in the United

        *      The study is positioned as the product of a "research
               team" at Carnegie Mellon University.@  It is described
               throughout as the "Carnegie Mellon study" and it is
               frequently mentioned that the "research team" estimated
               this percentage or counted that set of items.  Yet,
               nowhere is it mentioned that Rimm was an undergraduate
               electrical engineering student at CMU at the time the
               study was performed.  Instead, Rimm is listed as a
               "Researcher" and "Principal Investigator." (Note that the
               four funding sources are identified only as coming from
               Carnegie, but the type and kind of grants these are is
               not revealed.) This positioning capitalizes upon the
               reputation of Carnegie Mellon University, lends an air of
               authority and credibility to the paper, and increases
               Rimm's own authority by association.  

               The article is sole-authored. Not a single member of the
               extensive research team shared in the credit for the
               authorship of this paper.@ Given established standards of
               authorship as ownership of intellectual property in the
               academic and scientific community, we can only infer from
               this that no one on the "research team" felt their
               contributions merited the significance of shared

*       Manipulation

        *      The study was not subjected to peer-review.  The
               manuscript was deliberately "embargoed" for at least six
               months prior to publication, and was not made available
               to interested researchers.  This is highly unusual.  The
               paper was submitted to a law journal which is not peer-
               reviewed, despite the fact that it probably would be more
               appropriate in a behavioral science or public policy
               journal (most of which are peer-reviewed).  Since law
               journals have no one on the board to evaluate the merits
               of the methodology and likely not even the distinctions
               among BBSs, Usenet news groups, the Web, and the
               Internet, we offer the following hypothesis:  did Rimm
               place his article somewhere where it would appear
               credible and go unchallenged?

               At some point, an agreement was negotiated in which Time
               magazine obtained an advance copy of the manuscript in
               exchange for an "exclusive."  This was used in
               preparation of the July 3, 1995 Time cover story written
               by Philip Elmer-DeWitt. Given the vast array of
               conceptual, logical, and methodological flaws in this
               study, documented thoroughly below, Time magazine behaved
               irresponsibly in accepting the statements made by Rimm in
               his manuscript at face value.  Time had a responsibility
               to its readers to do its own peer reviewing, despite the
               embargo.  Indeed, Time reporters were made aware that the
               study appeared to have serious conceptual, logical, and
               methodological flaws that Time needed to investigate
               prior to reporting its story.  If Time was not able to
               evaluate the manuscript on its own, Time should have held
               the story until the manuscript was publicly available, so
               that expert opinion could have been solicited, or sought
               its own panel of objective experts for a "private" peer
               review.  In this way, Time would likely have recognized
               the study for what it was and not what it purported to be
               and prepared a balanced, critical report on the subject
               of digital pornography. Instead, Time presented, around
               lurid and sensationalistic art, an uncritical and
               unquestioning report on "cyberporn" based on Rimm's
               flawed study. This has had the extremely unfortunate
               effect of giving the study an instant credibility that is
               not warranted nor deserved and fueling the growing
               movement toward first amendment restrictions and

        *      The study appears to be driven by an underlying political
               agenda.  It is difficult to read the paper in its
               entirety and not come away with the conclusion that it is
               written in a manner which provides policymakers with the
               ammunition they need to obtain support for legislation
               that would censor certain types of information on the
               Internet and other emerging media.
*       Lack of Objectivity 

        *      Rimm makes numerous unsubstantiated causal statements. 
               These causal statements are not supported by the data. In
               many cases, the causal statements are inflammatory and
               outrageous. Sometimes they are ridiculous.  Additionally,
               data are often interpreted in a biased and selective
*       Methodological Flaws

        *      The article is rife with methodological flaws, several of
               them extremely serious. The origins of many numbers
               presented in the article are difficult, if not
               impossible, to determine. Much greater attention is paid
               to sensationalistic and inflammatory descriptions of
               image files, for example, than accurate descriptions of
               survey methodology.  In fact, in many cases important
               aspects of the methodology are simply not described at
               all. Methodological details are either omitted entirely
               or presented in such sparse detail that it is impossible
               for other researchers to 1) determine what Rimm actually
               did and 2) replicate the results.
        *      The study contains numerous discrepancies that cannot be
               resolved and raises a series of fundamental procedural,
               analytic, and implementation questions that can only be
               addressed outside of the article itself.  
        *      Operational definitions of "pornography" are ad-hoc,
               inconsistent, and misleading.

        *      Much of the data presented is consistently
               misinterpreted, particularly the Usenet data.

        *      The paper describes the results in a confusing manner
               which makes it very difficult to determine what Rimm
               actually did.  The manuscript is way too long and
               rambles.  It is organized in such a manner as to obscure
               the methodological issues.  This makes it difficult for
               the casual reader to draw his or her own conclusions
               about the merits of particular results from the study. 
               For example, discussions of Usenet readership at a single
               university are interwoven with worldwide Usenet
               readership statistics.  This is confusing and makes it
               easier to misinterpret his results, thinking that he
               might be talking about Usenet in general when in fact he
               is only talking about readership at a single university. 
               Definitions of online media are similarly presented in
               such a way that the reader is likely to draw the
               conclusion that BBSs, Usenet news groups, the World Wide
               Web, and the Information Superhighway are all one and the
               same, and what applies in one domain, is relevant to all.
        *      Rimm makes numerous unsubstantiated leaps of faith in his
               logical arguments.

        *      The research methodology is not up to the rigorous
               standards of a peer-reviewed journal.

        *      The study procedure raises a number of troubling ethical


Specific Comments             

Title and acknowledgement (p. 1849)

        *      The article's title states it concerns the marketing of
               pornography on the so-called "information superhighway,".@
               yet it appears in a law journal that is, by custom, not
               rigorously peer-reviewed.  The acknowledgement@ indicates
               that organizations and experts in pornography were
               consulted (but not listed), but no organizations or
               experts conversant in marketing research, survey
               methodology, and marketing on the Internet and  related
               online markets appear to have been consulted.

I. Overview (pp. 1849-1864)

        *      Rimm:          "The ... study adopts the "definition"
                              utilized in current everyday practice by
                              computer pornographers.  Accordingly,
                              "pornography" is defined here to include the
                              depiction of actual sexual contact
                              [hereinafter "hard-core"] and depiction of
                              mere nudity or lascivious exhibition
                              [hereinafter "soft-core"]...Accordingly, data
                              was (sic) collected for this article only from
                              bulletin board systems (BBS) which clearly
                              marketed their image portfolios as "adult"
                              rather than "artistic."  Any BBS or World Wide
                              Web site which made even a modest attempt to
                              promote itself as "artistic" or
                              "informational" was excluded." (fn. 1)@

               Rimm's definition of pornography is central to his study. 
               It is therefore reasonable to expect a detailed analysis
               of what pornography is, along with arguments for how it
               may be defined and measured.  Such discussion would
               include the advantages and disadvantages of each
               measurement approach and lead to a reasoned position of
               the operational definition employed in the study.  

               Because the results of his study depend on his definition
               and measurement scheme, it is surprising that the
               definition he proposes is so weakly supported, and fluid
               besides.  For example, in analysis to follow, Usenet
               newsgroups appear to be classified as "pornographic" if
               they contain the word "sex" in the title (except for
               alt.safe.sex), or if he judges them to be so.  Further,
               the footnote is misleading, because it implies that Rimm
               studied the Web with the same energy that he applied to
               adult BBSs, when in fact, he only searched the Web in
               order to locate and provide a simple count of sites
               judged to be "sex-related." (Appendix C, p. 1923 ff).@
        *      Rimm:          It is essential to note that Usenet and the
                              World Wide Web are merely different
                              protocols." (p. 1869)@

               This statement is erroneous and suggests a disturbing
               misunderstanding of the nature of online media,
               particularly as they relate to consumers and providers. 
               For definitions and discussion, see Hoffman and Novak's
               paper on Marketing in Computer-Mediated  Environments 

        *      Rimm:          "...this article discusses only the content
                              and consumption patterns of sexual imagery
                              currently available on the Internet and
                              "adult" BBS..." (fn. 2)@

               This statement is misleading because, in fact, the
               article discusses the content analysis of descriptive
               listings of images obtained from adult BBSs and the
               readership data from selected Usenet newsgroups. Usenet
               readership data can only tell that a Usenet group was
               accessed, but does not tell if any text files were read
               or any images were downloaded.
        *      Rimm:          "Every time consumers log on, their
                              transactions assist pornographers in compiling
                              databases of information about their buying
                              habits and sexual tastes.  The more
                              sophisticated computer pornographers are using
                              these databases to develop mathematical models
                              to determine which images they should try to
                              market aggressively." (p. 1850-51)@

               Every time consumers log on to what?  In the final
               analysis the article provides very little evidence, other
               than anecdotal or case study, to support the idea that
               pornographers are engaging in such activities. 

        *      Rimm:          "Computer pornographers are also moving from a
                              market saturation policy to a market
                              segmentation, or even individualized,
                              marketing phase." (p. 1851)@

               The statement is misleading because it implies that
               pornographers have a strategic policy which is now
               shifting.  However, the article supplies no evidence of
               the original policy, let alone the shift to a market
               segmentation strategy. 

        *      Rimm:          "It is clear that pornography is being
                              vigorously marketed in increasingly
                              sophisticated ways and has now found a
                              receptive audience in a wide variety of
                              computer environments." (p. 1852)@

               The article supplies no evidence that pornography is
               being "vigorously marketed," nor does it define
               marketing.  The study does not investigate audience
               receptivity in a "wide variety of computer environments."
               Instead, it studies download records from selected adult
               BBSs in the United States and Usenet postings (but not
               Usenet downloads).  Thus, this conclusion cannot be
               supported from the research presented in this paper. 
        *      Rimm:          "'Information Superhighway' and 'Cyberspace'
                              are used to refer to any of the following:
                              Internet, Usenet, World Wide Web, BBS, other
                              multimedia telephone, computer, and cable
                              networks." (fn. 7)@

               These two definitions are misleading and do not conform
               to commonly understood meanings of the terms by
               researchers and experts in the field.  

        *      Rimm:          In the top paragraph on page 1853, Rimm argues
                              that his study is the first to systematically
                              examine "pornography on the Information
                              Superhighway," and that it is now possible to
                              obtain "vast amounts of information about the
                              distribution and consumption" of pornography
                              on a much larger scale than previously

               Are there any previous studies of pornography on the
               "Information Superhighway," even if unsystematic?  In
               what ways does this study have to do with the
               "Information Superhighway?"  A framework should be
               developed for adult BBSs - the focus of this study - in
               the context of the "information superhighway."  For
               example, what percent of traffic do adult BBSs represent
               of the total "highway?"  What percent of users of the
               "highway" use adult BBSs?  What do the distributions look
               like nationally and internationally?  And so on.

        *      Rimm:          "...it maybe be difficult for researchers to
                              repeat this study, as much valuable data is no
                              longer publicly available." (fn. 9)@

               This is an astonishing and intellectually suspect
               statement, almost transparent in its effort to set up a
               case that this study cannot be falsified.  If subsequent
               research shows disagreement with the results of this
               study, Rimm can discount such results by saying that it
               could not be repeated anyway.  Instead, good scientific
               practice demands Rimm work to show how the study can be
               replicated by subsequent researchers.  However, as
               analysis below argues, even the analyses here cannot be
               replicated because Rimm provides no details of
               methodology which would enable that to happen.

        *      Rimm:          The first full paragraph on page 1853
                              discusses the 917,410 "pornographic" items
                              downloaded 8.5 million times that form the
                              bulk of the study.)@  

               Subsequent sections of the paper show that this paragraph
               is misleading in the extreme, as is the article title. 
               The title of the article suggests the research will
               concern a "survey of 917,410 images, descriptions, short
               stories and animations downloaded 8.5 million times." 
               (Note that Rimm does not perform "survey research" in
               this study, as no one is surveyed.)  

               On page 1853, the 917,410 items are broken down as: 

               *       450,620 items downloaded 6.4 million times from 68
                       adult BBSs
               *       75,000 items with an unspecified number of
                       downloads from 6 adult BBSs
               *       391,790 items with no download information from 7
                       adult BBSs

               These items include images, animations, and text files. 
               Rimm says that 10,000 "actual images" were "randomly
               downloaded" from adult BBSs, the Usenet or CD-ROM and
               used to verify the accuracy of the descriptive listings. 
               Rimm does not, however, 1) report the methodology used to
               randomly select the images, 2) provide frequency
               distributions of the images across the media they were
               obtained from, 3) specify the exact media used to obtain
               the listings (e.g. which CDROMS?), nor 4) indicate how
               the accuracy verification procedure was performed.

               In footnote 10 on page 1853, Rimm says the original
               number of downloads was counted at 6.4 million and that
               "a total of 5.5 million downloads are analyzed here."
               (emphasis ours).  He explains that the other "0.9 million
               concern animations, text, and other miscellaneous files"
               which he presumably excluded from analysis. He continues
               that "an additional 2.1 million downloads was later
               obtained from...Amateur Action BBS. In this way, the
               total number of downloads tabulated is 8.5 million."@

               We note that this tabulation of 8.5 million downloads is
               misleading for two reasons: 1) Rimm did not specify the
               period of time in which the 8.5 million downloads
               accumulated.  Was it one month?  One year?  Five years? 
               Ten years?  2) Rimm did not actually analyze 8.5 million
               unique downloads, as at least some were apparently
               excluded from analysis.

               While 8.5 million exposures to pornographic images may
               sound like a large number, let us put it into
               perspective.  Suppose a pornographic newsstand magazine
               had a circulation of 500,000, including subscriptions,
               newsstand sales and pass-along readership.  If there were
               10 pornographic photographs in a single issue of this
               magazine, there would be 5 million "exposures" in this
               single issue alone.  Thus, 8.5 million must be set in a
               context which specifies the time period, and the
               equivalent exposures in "competing" media during this
               time period. 

        *      Rimm:          "A total of 292,114 image descriptions
                              remained and are discussed here.  At least 36%
                              of the images studied were identified as
                              having been distributed by two or more "adult"
                              BBS." (p. 1854)@
               Apparently, Rimm analyzed 292,114 descriptive listings of
               images only, presumably representing 5.5 million
               downloads.  No indication is given of how duplicates were
               identified as such, nor distributed across the listings,
               either by individual adult BBS or by, for example,
               geographic region.

               In footnote 11 on page 1854, Rimm suggests that whatever
               method was used to identify duplicates had its validity
               confirmed by randomly sampling 100 "suspected
               duplicates," and presumably examining them.  Yet, he does
               not indicate how he "suspected" them in the first place,
               how they were sampled, and how the validity was
               "confirmed," as no details or statistics are provided to
               support the statements.)@
        *      Rimm:          Part II of this article addresses three issues
                              concerning pornography on the Usenet: (1) the
                              percentage of all images available on Usenet
                              that are pornographic; (2) the popularity of
                              pornographic boards in comparison to non-
                              pornographic boards, at both a university
                              studied and worldwide; and (3) the origins of
                              pornographic imagery on the Usenet. (p. 1854)@

               Our critique of Part II will show that due to serious
               methodological flaws, the study does not, in fact,
               provide accurate data on these issues. 

        *      Rimm:          All BBS data was (sic) collected in May and
                              June 1994, unless otherwise noted. (p. 1855)@

               In footnote 15 on page 1856, Rimm states that the study
               "tracks image repertoires over a fourteen-year period."@
               No clarification is provided here or subsequently to
               reconcile the discrepancy between the two-month data
               collection period and the 14 years or to illuminate on
               how the study follows "image repertoires" longitudinally.

        *      Rimm:          "...this study focuses entirely upon what
                              people actually consume, not what they say
                              they consume; it thus provides a more accurate
                              measure of actual consumption." (p. 1855)@

               Rimm analyzes aggregate download counts of descriptive
               listings of images available on adult BBSs.  Although
               download patterns would be expected to correlate with
               actual consumption (i.e. viewing), we do not know the
               extent to which individuals actually looked at the images
               (or, indeed, whether they looked at all). These
               limitations are not addressed in the study and no
               thoughtful discussion of the consumption experience is
               ever provided.  Further, absolutely no download behavior
               on Usenet news groups was ever examined by Rimm.
        *      Rimm:          Because the data is (sic) in many respects
                              exhaustive, statistical techniques and
                              assumptions that are commonly invoked to
                              impute general consumer behavior are not
                              necessary for this dataset.  Thus the research
                              team considers the inferences drawn highly
                              robust." (p. 1856)@

               Are the data really exhaustive of adult BBS?  Rimm does
               not provide evidence that the listings obtained from the
               BBSs represent a census.  Further, the statement that
               statistics are not necessary for these data is
               astonishing.  No evidence, statistical or otherwise, is
               ever provided in the article that the inferences drawn
               from these data are, indeed, "robust." 

        *      Rimm:          "The ... study examine 917,410 images, image
                              descriptions, short stories, and short
                              films..." (fn. 15)@

               Yet earlier on page 1854 and footnote 10 on page 1853,
               Rimm suggests he deleted all but the images from the
               database under consideration and retained 292,114 for
               "discussion."  Thus, how many items did the study
               actually examine?

        *      Rimm:          "The study results suggest a tremendous rift
                              between the sexual activities in which
                              Americans claim to engage, as reported most
                              recently by the study Sex in America, and the
                              sexually explicit activities presented in
                              images that many Americans consume." (p. 1857)@

               This statement is misleading, because Rimm did not study
               individuals, but aggregate download counts of descriptive
               listings of images available on adult BBSs.  The Sex in
               America study surveyed the general population, and did
               not examine individuals' consumption behavior as measured
               in downloads on adult BBSs in the United States. In other
               words, the two studies examine two completely different
               populations. Thus, there is no basis for the conclusion
               that a "tremendous rift" exists and the statement
               represents an "apples and oranges" comparison.

        *      Rimm:          "Among the ultimate findings of this study are
                              that digitized pornographic images are widely
                              circulated in all areas of the country and
                              that due to market forces, digitized
                              pornographic images treat themes...which are
                              not otherwise widely available." (p.1857)@

               The conclusion is not supported by the data because Rimm
               examined only downloads of pornography on adult BBSs and
               readership statistics of selected Usenet newsgroups.  He
               did not examine the distribution or consumption of
               pornography, by category or otherwise, in other media,
               nor does he provide evidence from others' examination. 
               Thus, there is no basis for the comparison.
        *      Rimm:          "One of the more intriguing questions raised
                              by this study is whether the general
                              population will demand the same types of
                              imagery currently in high demand among
                              computer users." (p. 1857)@

               This statement is misleading.  All computer users?  Some
               computer users?  How many are "demanding" it now?  What
               types?  Indeed, why would the general population be
               expected to exhibit the same types of preferences as
               subscribers to adult BBSs, which is the only group of
               "computer users" for which Rimm studied imagery?

        *      Rimm:          "The widespread availability of pornography on
                              computer networks may have a profound effect
                              on those who wish to utilize the emerging
                              National Information Infrastructure for non-
                              pornographic purposes." (p. 1858)@

               This statement is blatantly biased.  Rimm did not examine
               the extent of pornography on "computer networks" such as
               the Internet or online services, and provides no
               discussion, nor references to balanced discussion of
               these issues. 
        *      Rimm:          "While it may not be possible in the next
                              decade for such technology to automatically
                              classify images with the same precision as the
                              Carnegie Mellon linguistic parsing
                              software..." (fn. 21)@

               The precision of the noted software is never established,
               let alone described in any detail.

        *      Rimm:          "More than two dozen faculty, staff, graduate
                              and undergraduate students at Carnegie Mellon
                              University contributed in some manner to this
                              study." (p. 1861)@

               Yet in fact, the article is a sole-authored study,
               performed when the author was an undergraduate student in
               Electrical Engineering, that was not subjected to the
               usual rigors of peer-review and revision that are common
               for this type of research.  No one, other than Rimm, has
               accepted responsibility for the intellectual property in
               this study.  Further, the individuals listed on page 1849
               represent an acknowledgement by Rimm, rather than an
               endorsement by all of them of the manuscript.

        *      Rimm:          "After a year of exploring the Internet,
                              Usenet, World Wide Web, and computer Bulletin
                              Board Systems (BBS), the research team
                              discovered that one of the largest (if not the
                              largest) recreational applications of users of
                              computer networks was the distribution and
                              consumption of sexually explicit imagery." (p.

               As we continue to note, Rimm's study concerns download
               patterns on selected adult BBSs and readership statistics
               on selected Usenet newsgroups.  Rimm may have explored
               these systems, but provides no evidence for the
               conclusion stated above.  Further, Rimm's statement is
               misleading, as it implies that the largest recreational
               application is not just in downloads (i.e.
               "consumption"), but also in uploads (i.e.
               "distribution").  Rimm's study does not examine uploads.

        *      Rimm:          An unusual amount of data was (sic) freely
                              available from commercial "adult" BBS,
                              primarily as a consequence of the evolution of
                              the online industry.  Large commercial BBS
                              such as American Online, Compuserve, and
                              Prodigy do not carry hard-core pornographic
                              imagery, either for legal or policy reasons. 
                              As a consequence, several thousand
                              comparatively small "adult" BBS have sprung up
                              across the country."  (p. 1861)@

               The statements are misleading because no evidence is
               provided to support the conclusion of a causal link
               between activities on commercial online services and
               adult BBSs.  

        *      Rimm:          "In many instances, the research team was able
                              to persuade the owners of these BBS to provide
                              information about subscriber consumption
                              habits." (p. 1862)@

               This is a troubling statement.  How was Rimm able to
               obtain such consent?  Was it "informed consent?"  Did
               Rimm provide full disclosure to these operators about the
               nature and objectives of his study?  Did Rimm "debrief"
               them afterwards?  Did he get the permission of the
               subscribers of these BBSs to examine information about
               their consumption habits?  Did Rimm submit a proposal of
               his methodology for such "persuasion" to the University
               Human Subjects Committee?  Did they approve the research
               and the methodology?
II. Usenet (pp. 1865-1876)

        *      Rimm:          "This article will first discuss the
                              methodology and results of the study of Usenet
                              images and will then explain the methodology
                              and results of the study of BBS images." (p.

               Footnote 28 (p. 1865) appended to this sentence refers
               the reader to footnotes 25-27 "for discussion of the
               distinction between the Usenet and commercial BBS."  Such
               distinctions are critical for correct interpretation of
               Rimm's results and do not belong in footnotes. 
               Nevertheless, examination of the footnotes reveals the
               following: footnote 25 "assumes that the reader has a
               basic understanding of Usenet and BBS." (p. 1862), and
               refers to reader to several books and a magazine;
               footnote 26 cites a 20-year old FCC document on "MTS and
               Wats," (p. 1863) and footnote 27 cites a brochure "on
               file with the Georgetown Law Journal" (p. 1864).                          
        *      Pornographic vs nonpornographic imagery in the
               alt.binaries groups
               Rimm:          Rimm states that he examined "[a]ll of the
                              Usenet newsgroups with the prefix
                              'alt.binaries'" from September 21-September
                              27, 1994 and goes on to say that "[t]he number
                              of new images posted each day was tabulated
                              for both pornographic and non-pornographic
                              newsgroups."  (p. 1865)@

               No rationale for excluding audio and text is provided
               other than they were "not the subject of this study." 
               Does it make sense to look at all types of pornography on
               the Usenet and compare that to all other types of
               information?  Rimm does not indicate how he determined
               which alt.binaries groups were pornographic and which
               were not.

               In what manner did Rimm control for duplicates, resent,
               or non-pornographic images?  Did Rimm counts posts or a
               complete image?  (Note that a single image could have up
               to 10 more files to make it complete.)  In effect, what
               was the unit of analysis: a post or an image?  On
               Saturday, 7/1/95, a colleague counted the number of posts
               on alt.binaries.pictures.erotica and found 1650 posts. 
               One image was 41 posts long and represented 2.5% of the
               message volume alone.  The article is moot on these
               important methodological details.

        *      Popularity of Pornographic vs NonPornographic Usenet
               Rimm:          "The research team was also able to examine
                              the online habits of 4227 users at a mid-
                              sized, private university in the northeast."
                              (p. 1865)@

               This raises troubling issues. How was Rimm able to
               conduct such examination? Did he obtain "informed
               consent" from each student?  Did Rimm provide full
               disclosure to these students about the nature and
               objectives of his study?  Did Rimm "debrief" them
               afterwards?  Did the University Human Subjects Committee
               approve this examination?   It is curious that Rimm
               argues in numerous places about the possible public
               policy implications of his work, but does not raise the
               ethical implications of conducting such research (only
               the implications of reporting it).  See, for example,
               footnote 40 on page 1869, where he discusses his decision
               not to report "detailed demographics of the university
               population of computer pornography consumers" but makes
               no mention of whether it is appropriate to gather the
               data in the first place.

        *      Rimm:          In footnote 30 on page 1865, Rimm argues that
                              the 11% of computer users at the private
                              university "block" site statisticians from
                              monitoring in order to "avoid detection" of
                              their online activities.  After discussing a
                              behavioral analysis of child molesters, he
                              proposes that "it is possible that some
                              Internet users who block their accounts prefer
                              sexual images of children and wish to avoid

               This argument is one of the more outrageous in the paper
               and represents an invalid causal link.  In the first
               place, there is no evidence that the 11% who "block"
               their activities are child molesters, and in the second
               place, there is no evidence that the 11% are
               representative of the broader population of Internet
               users.  Thus, there is no basis for the proposal that
               Internet users who do not wish their activities monitored
               prefer to look at "sexual images of children."

        *      Percent of pornographic imagery in Usenet binaries

               Rimm:          "Among the pornographic newsgroups, 4206 image
                              posts were counted, or 83.5% of the total
                              posts."  (p. 1867)@

               The interpretation is incorrect and the number is grossly
               inflated.  It is based upon 17 alt.binaries groups that
               Rimm considered "pornographic" and 15 alt.binaries groups
               that Rimm considered "non-pornographic."  However, Rimm
               does not provide a listing of the names of these groups,
               no distributions of posts in these groups, and no
               methodological discussion of how he counted and
               determined posts were either pornographic or not, so
               there is no objective evidence of whether these groups
               are, in fact, "pornographic."  

               Also, no information is provided on the degree to which
               these 32 groups comprise the complete universe of Usenet
               imagery.  Further, as the methodology for counting the
               number of images is not specified, it is likely that even
               given Rimm's definitions and selection of 32 groups, the
               percentage is inflated due to the inclusion of
               non-pornographic next comments and multi-part images in
               the counts.  What are the distributions of posts, by type
               of post (imagery, text, audio) in each of these
               newsgroups?  What were the total numbers of posts to each
               group and to each set of groups and to Usenet overall
               during the period?  How did Rimm determine that the 4206
               image posts to the 17 supposed pornographic alt.binaries
               groups are, in fact, pornographic?

               A more accurate interpretation is that of 83.5% of the
               images posted to 32 alt.binaries newsgroups came from 17
               groups that Rimm determined were pornographic.

               To make matters worse, Rimm grossly overgeneralizes his
               results in footnote 36 (p. 1868) and his summary (p
               1914):  "83.5% of all images posted on the Usenet are
               pornographic."@  This is a particularly misleading
               misinterpretation of his narrow result.                            

        *      Misleading interpretation of "popularity" of types of
               Usenet newsgroups.  

               Rimm:          (p 1849) "'Pornography' is defined here to
                              include the depiction of actual sexual
                              contact...and depiction of mere nudity or
                              lascivious exhibition."@
               Rimm uses bold text to identify "newsgroups identified as
               having pornographic content" in Table 1 and Table 2.
               Included among pornographic newsgroups are "alt.sex" and
               "alt.binaries.pictures.supermodels."  This is not
               consistent with Rimm's stated definition of pornography,
               as there is little of what would be considered
               pornographic content in these groups. It is a biased and
               inflammatory characterization of these Usenet groups.  

               The column headings in Table 1 are not explained.  Is the
               user base 4227 from page 1865 or some other number?  This
               particular site receives only 3600 (p. 1870) of the
               14,000 Usenet newsgroups (p. 1862) or only 25.7% of all
               groups.  This seems like a small percentage of total
               groups.  Is it?  What do the percentages at other
               institutions look like?  Without knowing this, it is
               difficult to generalize beyond this site to the entire
               Usenet domain.  What would happen if we included data
               from the other 10,000+ sites?

               It is truly astonishing that there are no .comp or .news
               groups in the Top 40 Usenet news group at the university
               studied.  Indeed, if the university is Carnegie Mellon,
               this is simply unbelievable.  By this chart, only 99
               readers are required in order to put it at number 40. 

               Additionally, the Top 40 newsgroups in Table 1 differ
               dramatically from the Top 40 overall, according to the
               arbitron statistics. 

               Rimm:          In footnote 30 on page 1865, Rimm argues that
                              "there is no reason to believe consumption at
                              the university study differs from that of
                              other universities from which pornographic
                              Usenet newsgroups can be accessed."@

               But, in fact, there are reasons to believe otherwise.  A
               study conducted at Vanderbilt University (Varki 1995) as
               part of the requirements for a doctoral seminar on
               "Marketing in Computer-Mediated Environments" showed that
               the top Usenet news groups in terms of number of postings
               differs markedly and in important ways from the Top 40
               list presented in Table 2 (p. 1872).  Since this is a
               worldwide listing, intuition alone would suggest the
               likely presence of regional differences, at the least. 
               In any event, no evidence is presented to support his
               reasoning in footnote 30.

               All of these problems suggest that the university in
               question may actually be fairly atypical in its use of
               Usenet newsgroups, which limits its generalizability.
               Rimm:          "In broad terms, the research indicated that
                              pornographic newsgroups are accessed more
                              frequently during the school year than during
                              summer recess.  This suggests that, in
                              comparison to teachers, faculty, and staff, a
                              disproportionately large number of students
                              access Usenet pornography." (p. 1969-1870)@

               In fact, the conclusion does not follow since Rimm does
               not present evidence (e.g. counts, frequencies, and
               proportions) indicating how many students access
               pornography relative to the other groups.  Rimm does not
               provide a version of Table 1 for the academic year, so
               that readers may draw their own conclusions.
               Rimm:          "The fact that alt.sex.stories is currently
                              more popular than
                              alt.sex.pictures.binaries.erotica has been
                              often misinterpreted as an indication that
                              stories are more popular than images." (p.

               In fact, Rimm presents no evidence that such
               "misinterpretation" exists, although we can assume the
               interpretation exists.  His alternative explanation is
               interesting, but no data are offered on how many users
               are discouraged by the level of technical sophistication
               required to access these groups.  Indeed, a rival
               hypothesis is that these groups are accessed by a
               singularly technically sophisticated user, not the
        *      Percentage of sites containing "pornographic" Usenet

               Rimm:          (p 1871) "The worldwide statistics suggest
                              that Usenet hosts appear less willing to offer
                              their readers access to pornographic
                              newsgroups than other types of newsgroups. 
                              81.2% of the sites offer access to non-
                              pornographic newsgroups, whereas only 55.8% of
                              the sites offered their readers access to the
                              pornographic newsgroups."@

               In our opinion, Rimm has clearly misinterpreted the data. 
               An examination of Tables 2 and 3 will immediately reveal
               that the important distinction is not between
               "pornographic" and "non-pornographic" groups, but between
               "alt" and "non-alt" hierarchies.  All Rimm's
               "pornographic" groups are from the "alt" hierarchy.  No
               alt group in Tables 2 and 3 is carried by more than 66%
               of sites.  While alt.binaries.pictures.erotic (one of
               Rimm's "pornographic" groups) is carried by 53% of sites,
               alt.binaries.pictures (a "non-pornographic" group) is
               carried by only 49%, and alt.binaries.sounds.tv is
               carried by only 34% of sites.  

        *      Misleading portrayal of newsgroup readership.

               Rimm:          (p 1873) "The newsgroups are ranked in Table 2
                              by the estimated total number of readers

               Rimm identifies Brian Reid's "arbitron" script as the
               source of the data in Table 2.  However, Rimm does not
               provide Reid's caveat on exactly what "readership" really
               means.  Reid (Usenet Readership Summary Report for May
               95) is careful to note that:

               "To 'read' a newsgroup means to have been presented with
               the opportunity to look at least one message in it." ...
               "Assuming that 'reading a group' is roughly the same as
               'thumbing through a magazine', in that you don't
               necessarily have to read anything, but you have to browse
               through it and see what is there."

               This is a critical point.  There is absolutely no
               information from Table 2 on how many of the 260,000
               "readers" of alt.binaries.pictures.erotica actually
               downloaded and uudecoded a binary image file.  The
               arbitron data is not tracking downloads.  In fact, it
               would be completely consistent with Reid's definition of
               readership if none of the "readers" of
               alt.binaries.pictures.erotica ever saw a pornographic

               Thus, the results shown in Table 2 simply cannot be used
               to establish the exposure of "readers" to pornographic
               imagery.  A reasonable hypothesis is that "readers" are
               simply curious about what is in these groups, and browse
               the titles to get some idea.  As Rimm notes, decoding
               Usenet binaries requires a non-trivial degree of
               technical skill.  

               We should further note that if one takes the estimate of
               individuals with Internet access as 20 million, then at
               most we are speaking of about .1% of Internet users
               accessing the alt.binaries.pictures.erotica newsgroup,
               and almost surely, the percentage actually downloading
               and uudecoding pornographic images is much lower than
               even this very low percentage.

               Rimm:          In footnote 31 on page 1866, Rimm suggests
                              that the (presumably total) number of readers
                              of alt.binaries.pictures.erotica on Usenet is
                              260,000 per month.@

               Rimm provides no discussion of the methodological details
               necessary to understand this estimate.  How is this
               number estimated?  How are multi-part image files
               counted?  How are robot extractions handled? Are these
               260,000 people unique?  Or, could they possibly
               represent, for example, the same 9000 individuals per day
               for 30 days?  How does the "arbitron" script keep track
               of individual users?  In other words, are reach and
               frequency confounded?   Does Rimm know?

        *      Amount of pornography in Usenet groups.

               Rimm:          "Of this 11.5%, approximately 3% [of messages
                              on the Usenet] is associated with Usenet
                              newsgroups containing pornographic imagery."
                              (p. 1869)@
               Rimm fails to take these traffic percentages to their
               logical conclusion, which is that less than 1/2 of 1% (3%
               of 11.5%) of the messages on the Internet are associated
               with newsgroups that contain pornographic imagery. 
               Further, of this half percent, an unknown but even
               smaller percentage of messages in newsgroups that are
               "associated with pornographic imagery" actually contain
               pornographic material.  Much of the material that is in
               these newsgroups is simply text files containing comments
               by Usenet readers.  

               Rimm:          (p 1873) "Moreover, 20,644 of the 101,211
                              monthly Usenet posts in the top forty
                              newsgroups, or 20.4%, are pornographic.@

               This figure is inflated and incorrect.  Rimm is assuming
               that 100% of the content of the so-called "pornographic"
               newsgroups in Table 2 is pornography.  But, this is
               obviously incorrect.  A large number, if not the
               majority, of messages in these groups are simply text
               representing discussion and comments - not pornographic.
               In addition, large images are typically broken into
               multiple parts, so that one large .gif file might
               actually consist of ten or more physical files.  Further,
               even single file images often have a separate descriptive
               header (which should be considered non-pornographic). 
               While it is impossible to determine from the results Rimm
               has presented what proportion of monthly Usenet posts are
               "pornographic," we can safely conclude that the
               percentage is far below what Rimm states.  

        *      Origins of pornographic imagery on the Usenet. 

               Rimm:          (p 1874) "71%, or 1671 of the 2534
                              pornographic images downloaded from the five
                              Usenet newsgroups studied over a four month
                              period, originated from "adult" BBS." @

               This is a critical percentage, yet we question its
               validity.  Virtually no support is given for this
               percentage other than Rimm's in the text statement that
               1671 images originated from adult BBSs.  We cannot
               determine how Rimm arrived at this number from our
               reading of the manuscript.   Is it an estimate?  A count? 
               How was it estimated or counted?
               Rimm lists the five Usenet newsgroups on which he says
               "[t]he largest selection of sexual imagery was
               discovered" at the northeastern university (p. 1866)@ and
               notes in footnote 32 (p. 1867)@ that these sites were the
               largest available at the "research site."  He further
               says that between "April and July of 1994,...all
               available images (3254) [were downloaded] from these five
               newsgroups." (p. 1867)@

               There must be a typographical error, because earlier Rimm
               stated that the alt.binaries groups were not examined
               until September of that year, so it cannot be possible
               that months earlier he was able to determine the groups
               with the largest selection of sexual imagery.  The
               appearance of alt.sex.fetish.watersports is also
               confusing since it is not an alt.binaries group.  It is
               possible that it all makes sense, but it is very
               difficult to sort out from the confusing exposition.  We
               also wonder if group size and availability are confounded
               with amount of imagery.
               The main issue is that convincing evidence has not been
               presented that these five groups contain the largest
               selection of sexual imagery.  Where did this list come
               from? These groups are at a single university site. Was
               a systematic analysis of all Usenet groups performed to
               generate this list?

               Did Rimm control for duplicates, resent, or non-
               pornographic images?  Did Rimm counts posts or a complete
               image?  What was the unit of analysis: a post or an

               Rimm states that the images from the five Usenet groups
               were classified into three categories (p. 1867): 1)
               images originating from adult BBSs ("the name, logo, and
               telephone number of the BBS appeared next to or within
               the image."; 2) pornographic images which did not
               originate from BBSs; 3) "PG/R" images ("no sexual contact
               or lascivious exhibition."). 

               Curiously, there is no category for images were are not
               pornographic!  Was every single image on these groups
               pornographic? Rimm does not indicate whether these
               categories are mutually exclusive; for example, how were
               "PG/R" images with a BBS logo counted?

               In any event, Rimm states (on page 1867) that there were
               "a total of 2830 images for analysis," but does not
               report the frequency of images in each of the three
               categories.  (He states that 13% of the images could not
               be downloaded, which makes us wonder whether other
               figures presented need to be similarly adjusted to
               account for technical difficulties which must ultimately
               lower consumption rates.) However, seven pages later, the
               total number of pornographic images downloaded from the
               five groups shrinks to 2354 images, with no explanation! 
               If we accept the 1671 as indicative of the number of
               images in those five groups that Rimm determined came
               from adult BBSs, then the percent of images originating
               is 59% (1671/2830) if we use his first number and 71%
               (1671/2354), if we use his second.

               Rimm:          (p 1874-5) "For those who consider pornography
                              to include the additional 476 'PG' or 'R'
                              rated images defined in the methodology
                              section, 59% of all Usenet images originate
                              from 'adult' BBS."@

               Since adding counts to the numerator of a fraction must
               increase the resulting percentage, 71% followed by 59%
               cannot be correct.  Of course, perhaps this is a
               typographic error or an error of confusion.  Was the 59%
               meant for the first percentage reported using the first
               (and larger) denominator?  In that case, is the 71% also
               a typographical error or error of confusion?  In any
               event, this additional calculation makes no sense unless
               all images in the third "PG/R" category were exclusively
               from adult BBSs.  Thus, the percentage of 59%, which
               should be higher than 71% but must be a typo), is
               misleading until clarification can be provided!

               Note that the percentages are suspect (Which denominator
               is "right?" The larger?  The smaller?  Neither?  Are 1671
               and 476 even the correct quantities?), not only because
               of the confusing manner in which the information is
               presented, but also because of the more serious
               methodological criticisms made earlier about the
               selection of the five newsgroups on which these numbers
               are apparently based. 

III.  Pornographic "Adult" Commercial BBS (pp. 1876-1905)

        *      Number of adult BBSs examined 

               Rimm:          "...[T]he team either subscribed to, or logged
                              on as a new user or guest, to a number of
                              representative pornographic BBS and collected
                              descriptive lists of the files offered by
                              each." (p. 1876)@
               Rimm reports that Boardwatch estimates that 5% of BBSs in
               the country are "adult," (fn. 35, p. 1867) but does not
               report a figure on the total number of BBSs, only that
               5000 BBSs of any type were identified (p. 1877) and that
               500 "active" adult boards were located for further study. 
               Since this represents 10% of his list, we can assume that
               Rimm's list of BBSs was not complete.  Rimm indicates
               that "most" of these 500 adult boards were "chat" boards,
               and still others were "transient."  He gives no figures
               on how many comprised each category.

               Rimm:          "To the best of the research team's knowledge,
                              the BBS included in this study comprise most
                              of the medium- and large-sized "adult" BBS in
                              the country that existed at the time of the
                              research." (p. 1877)@

               Rimm does not indicate how many boards this represents,
               how they were sampled to be "representative" (p. 1876),
               whether the list of adult BBSs Rimm sampled from was
               exhaustive, or whether Rimm used his "judgment" in
               selecting BBSs or in generating the list of BBSs to
               sample from. 
        *      Number of descriptive lists examined 

               Rimm:          "This portion of the study analyzes a total of
                              450,620 files that are classified..." (p.

               Previously, Rimm indicated that 292,114 descriptive
               listings were retained for analysis.  How many listings
               were actually collected? How many pornographic images do
               these listings represent?  How many were movies?  How
               many were text files?  How many images were selected from
               the BBSs and how were they selected?  

               Rimm indicates that both "descriptive lists" of
               pornographic images as well as a "representative sampling
               of the images themselves" were collected from the BBSs
               (p. 1877).  Rimm does not say how many images were
               sampled, how they were sampled to be "representative," or
               what they were supposed to be representative of.
        *      Ethical lapse?

               Rimm:          "Members of the research team did not, as a
                              rule, identify themselves as researchers." (p.

               As before, this is troubling.  Why didn't Rimm identify
               himself and his research objectives to the operators? Did
               Rimm obtain permission to "collect" the information from
               the BBSs?  Did Rimm provide full disclosure to these
               operators about the nature and objectives of his study? 
               Did Rimm "debrief" them afterwards?  Did he get the
               permission of the subscribers of these BBSs to examine
               information about their consumption habits and report the
               cities they lived in (see Appendix D: pages 1926-1934)? 
               Did Rimm submit a proposal of his methodology for such to
               the University Human Subjects Committee?  Did they
               approve the research and the methodology?  Does Carnegie
               Mellon approve of publishing the cities that consumers of
               adult BBSs live in?  How did Rimm obtain the demographic
               information on adult BBS subscribers? (as noted on p.

        *      Results of the linguistic classification scheme

               Despite twelve pages of largely anecdotal discussion of
               the content analysis of the descriptive listings, the
               methodology is never once described formally, either in
               terms of the algorithm, or the software used to implement
               the algorithm.  In the scholarly literature it is not
               only customary to offer the software to those who wish to
               replicate your results, for some journals it is mandatory
               (as is making the data available).  Nowhere does Rimm
               indicate that the data or the software that categorized
               the listings are "available from the author."

               Validity and reliability are not established. This
               despite the fact that standard statistical procedures are
               available for determining reliability and validity.  The
               few numbers that are presented in this section are either
               poorly defined or not defined at all.  Other quantities
               are mentioned as being "high," but not reported (e.g. see
               footnote 70@ in which Rimm asserts that "[t]he
               presentation of kappa values...was considered unnecessary
               because of the high level of reliability."  Yet this
               "high level" is never reported).  Elsewhere, Rimm
               suggests that "validity was high,"@ (p. 1888), but it too
               is not reported statistically; or, Rimm states that he
               performed "a statistical analysis"@ on the data (p. 1894),
               but the type of analysis nor its results are not
               reported.  Such examples, which render the statements
               they are intended to support, meaningless, are too
               numerous to catalog here.

               Relatedly, numbers or data are not reported that would
               help the reader understand the analysis, and numbers that
               are reported are pursued for additional insights.  

               This section is ad hoc and weak; no reliable and valid
               conclusions can be drawn from the analysis as presented.
               Moreover, this is a standard content analysis problem. 
               Content analysis has a large and rich literature, yet
               there is not a single citation to the either that vast
               literature or the related areas of AI software, and
               classification and categorization.  
               As Rimm presents it, it is not possible to replicate the
               categorization he performed, let alone determine how he
               performed it.  Thus, the methodology and this entire
               section, are impossible to evaluate.  

               Numerous questions must be raised:  What time period or
               periods are represented in the listings?  Did Rimm
               control for time in his analyses?  Were the data adjusted
               to account for differing lengths of time of each listing? 
               For example, adjusting for date first posted on the BBS?
               Rimm's procedure implicitly assumes that all downloads
               are a function of consumer demand and no other variables. 
               What about availability of certain kinds of images?  The
               cost of the images?  Their size?  Consumer demographics?

               Rimm states that "[m]any BBS either hide [the listing]
               information from their customers or do not provide it."
               (pp. 1879-1880).@  But on page 1878, Rimm states that
               listings have a typical record structure which he
               diagrams in Figure 3.  If the information is hidden by
               "many" operators (how many hid it in his study?), or many
               do not provide it (how many did not provide it in his
               study?), how did Rimm get it?  Rimm suggests that
               operators were "persuaded" to provide the information
               "privately." (p. 1881) What does this mean?  How were
               they persuaded?  What is meant by "privately?"

               How valid are the sixty-three basic categories? Were the
               categories validated by judges?  Did human beings ever
               look at any descriptions to validate the classification
               scheme?  If so, how many?  

               What exactly was the procedure the judges went through as
               part of the classification process?  Rimm notes that
               "judges...were not avid consumers of pornography and thus
               did not recognize names of particular pornographic
               'stars."@ (p. 1886)  Did this lack of experience on the
               part of the judges affect or bias the classification
               procedure?  Typically, judged are chosen for their

               How was the "final precedence scheme" in Figure 6 arrived

               Given that Appendix A describes dozens of categories, why
               are percentages not reported for individual categories
               within the major groupings.  These percentages are
               important to know because some of the individual
               categories may be considered less extreme than others. 
               Without knowing the distribution of categories of images
               within each broad group, it is difficult to know what the
               group actually represents. 

               Rimm:          Thus, analyzing the data in these four classes
                              presents a highly reliable means of exploring
                              the explosive growth of pornography on the
                              Information Superhighway." (p. 1886)@

               Rimm never shows that his method of analysis is "highly
               reliable"  and Rimm never shows that the growth in
               pornography is "explosive," on the "information
               superhighway" or anywhere else.

               Page 1889 adds nothing to the reader's understanding of
               the methodology.  What is the point of including this

               On page 1890, Rimm notes that there were 35 adult BBSs. 
               How is this figure reconciled with the 68, 6 and 27 adult
               BBSs discussed on page 1853?@
               What is the point of including the Amateur Action BBS
               Case Study (pp. 1896-1905)?@ 

               In general, the conclusions Rimm makes are not supported
               by his analysis. Because the content analysis and
               classification scheme are "black boxes," because no
               reliability and validity results are presented, because
               no statistical testing of the differences both within and
               among categories for different types of listings has not
               been performed, and because not a single hypothesis has
               been tested, formally or otherwise, no conclusions should
               be drawn until the issues raised in this critique are

IV.  Conclusions
               Rimm:          "[A]ttempts at regulation may be ineffective;
                              "the net interprets censorship as damage and
                              routes around it" is a well known expression
                              among Usenet enthusiasts.  The current
                              structure of the Usenet requires that
                              individual sites choose between an "All things
                              not expressly permitted are prohibited
                              policy," or conversely, an "All things not
                              expressly prohibited are permitted."  A middle
                              ground does not appear viable.@

               Curiously, Rimm does not consider the alternative of
               user-imposed, rather than state-imposed controls.

This document has been accessed XXXX times and was last updated: Jul 7, 16:18
The number of accesses today is XXX

(c) Copyright 1994, 1995 D. Hoffman and T. Novak, Owen Graduate School of Management, Vanderbilt University, Nashville, TN 37203