My Criteria for Evaluating Bible Study Software
In evaluating Bible Study programs, I break out my analysis into several general areas of consideration:
- Their ability to support actual Bible study tasks (core functionality)
- The scope and quality of available Biblical texts and other core resources (resources that help establish what the Bible says)
- Their ability to integrate user-created content
- Their ability to integrate functionality and resources
- Program usability (interface, presentation, and management)
- The scope and quality of their catalog of secondary resources (resources which interpret what the Bible means)
- Technical quality and program performance
- Cost and Marketing
Evaluating Core Functionality
Software Bibles are not particularly good for reading, but they are great for studying, because of the ease the computer makes of searching, listing, indexing, sorting, grouping, comparing, linking, cross referencing, annotating, and collecting/publishing (e.g. exporting to a word processor). This is what I call the core functionality of Bible Study software. The primary purpose of a Bible Study program is to act as an electronic concordance. Its most fundamental and important function is to search Bibles, and display useful results.
Searching the Bible – Can you search by words, phrases, and references? Can you search for multiple terms? Can multiple search terms be logically related using Boolean logic (AND, OR, NOT, XOR), and organized using parentheses? Can you search for words NEAR each other? Can you search for terms that meet multiple criteria (e.g. all occurrences of a specific English word, but only if it translates a particular Greek word or Strong’s #)? Can multiple translations be searched simultaneously? Can you search for either particular word forms or for word families? Can you search for original language terms? Can you search for either lemma or particular forms? Can you search for morphologies? Can you search for grammatical constructions?
How accurate is the search engine? Does it consistently return every occurrence of the search term(s)? Are any false hits returned (e.g. extraneous text)? Is marginal and other editorial content excluded by default? Can it be excluded? Can publisher footnotes be searched?
How much control do you have over search range scope? Can you search in multiple, non-contiguous sections? How long do searches take to execute? Does the program have a searching tool optimized for searching the Bible (as opposed to a one-size-fits-all tool)?
Displaying and Working with Bible Search Results – Can you have multiple results sets open simultaneously? Are the search parameters identified in the results window (so you can see which search results you’re looking at)? Are search terms/results clearly identified within context? Are different search terms differentiated in results display (e.g. by different color highlighting)?
Can results be displayed as either refs or verses? Can contiguous verses be displayed as passages? Can results be displayed in Biblical context mode? Can results be shown in multiple translations (parallel)? Can you choose between showing hits from a particular translation and showing hits from all searched translations?
Are results statistics displayed? Do they include both number of hits and number of verses? Are statistics displayed for grouped subsets? Are results numbered/labeled for easy reference? Can search terms be aligned within results for easy comparison? Can a results set be grouped into subsets? Can results sets of word families be grouped by word forms? Can results sets of multiple search terms be grouped by term?
Are search results displayed as static text, or are they intelligent with program functionality (e.g. can you click on a KJV word in results to see Strong’s info on it, etc.)? Are they displayed in functional Bible windows? Can results be shown as either one entry per hit (concordance style) or one entry per verse (Bible style)? Can you change translations? If so, with or without verse granularity? Can results be re-ordered by the user? Can you save & retrieve results sets? Can results sets be annotated?
Can you modify results with manual adds or deletes? Can you search within results? Can results be modified by additional search criteria using AND, OR, NOT or XOR? Can you use Boolean Set Logic to combine and contrast (AND, OR, NOT XOR) different results sets? Can you copy results between results windows? Can results be copied to other windows? Can they be exported as either refs or verses?
Displaying Biblical Texts – Are paragraph and verse-by-verse viewing modes available? Can multiple translations be shown in a single window in parallel? How easy is it to add or change translations in a window? What layout options exist for parallel view? Can the translations be ordered by the user in parallel view? Are original language texts supported in parallel view?
Is embedded data identified (e.g. footnotes, xrefs)? Can embedded data be viewed in-line? Can in-line viewing be turned off? Are there other viewing options for this material? Can extra-biblical text be suppressed? Is there a composite view of notes and/or xrefs for multiple translations? Are existing user annotations identified in the window?
Can you view multiple instances of the same translation? Is a history maintained to facilitate easy “page-flipping”? How easy is it to jump to specific passages? How useful are other navigation controls? Is the window look clean or cluttered? Can the user control font choices and/or sizes? How easy is the text to read?
Exporting Biblical Text to Word Processor – Can selected text be copied with poetic/paragraph formatting? Can the user select (on the fly) which translation(s) to utilize in copy/paste? Can non-contiguous selections of text be exported? Can you copy references only (no text)? Can user and/or publisher notations be included in the copy?
How much control does the user have over reference formatting? Are citation format options available? Can citations and references be suppressed? Can they be suppressed on the fly, or does it require a change to application preferences? Can you send copies directly to a word processor in single step? Can pastes be initiated from within a word processor without the Bible Study application running?
Bibles and Primary Reference Resources
These are the basic resources we use to understand what the Word of God says.
What translations are available? Which (if any) are tagged with Strong’s #s? Are there original language texts available? Are they tagged with Strong’s #s and/or morphological information? Are proto-translations available (e.g. LXX, Vulgate)? Are interlinears available? Are Deuterocanonical books included when available in a translation?
Are translators’ and/or third-party Biblical xrefs available? What is available for lexicons and other original language word study resources? Are any of these works tagged to support linking with tagged texts? Are topic indexes available? What about English language tools, dictionaries & encyclopedias? Are there maps available? Are they interactive? Is there an integrated audio Bible?
Integrating User Content
A key element of truly useful Bible Study programs is their ability to integrate user-created content – which could be couched as the ability to easily work with the results of last month’s study as a resource to help put together today’s study.
Bible Annotations and Markup – Can users attach annotations to biblical texts? Are annotations linked to all translations (like a commentary), or translation-specific (like footnotes)? Are highlighters and other tools available for text markup? Are text markups translation-specific, or tied to the verse context? Can annotations be applied to passages (verse ranges)? Can user annotations be searched?
Can annotations be stored in multiple sets (notebooks)? Can notebooks of annotations be scrolled/navigated like a book, or are they strictly tied to Bible windows? Does the annotation tool identify the existence of other user annotations relevant to the verse context? How are annotations output for printing, or exporting to text files? Can annotations be easily shared within the user community? Can user annotations be used as an xref resource by the program?
User “Books” – Does the program support user-created commentaries, topically-tagged books, and/or outline books? Do these user creations behave like built-in resources of similar type? Can the user create truly custom reading plans? Can custom reading plans include extra-biblical reading? Can user “books” be easily shared within the user community?
Documents – Can the user create documents within the program? Will they persist as generally available resources, or are they tied to a specific “desktop”? Do they support organizing elements such as sections or tables? Will the program support multimedia in documents? Can the user use the program to create presentations (e.g. slides)? Can the program search user documents for words and/or refs? Are they available as xref resources? Can the user define their location within the file system? Can user documents be easily shared within the user community?
Verse Lists – How are user-defined sets of Scripture passages (e.g. Verse Lists) handled? Does their text support program functionality? Can multiple verse lists be used simultaneously? Can they be saved? Can they be shared? Can a search tool tell you which verse lists contain a particular verse? Can you perform text markups that remain specific to the verse list?
Is their structure flat or do they allow for grouping in sections? Can sections be nested in outline style? Can you copy between verse lists and/or sections, combine them, use set logic to join them (AND, OR, XOR, NOT)? Can they include commentary or annotation? To what scope can annotations be applied: entire verse list, section, verse/passage? Are the annotations searchable? Can an annotation to a passage be identified by an xref tool?
Can they display either verses or refs? Can contiguous verses be grouped into range passages? Can verse order be customized? Do they have translation flexibility? Do they support original language texts? Do they support interlinear text display? Can sections show verse counts of their content? Can sections be rolled up and expanded? How easy is it to get data out of these objects? How flexible are the printing options? Can the content be published to document format (html, rtf, pdf, doc, etc.)?
Resource Integration
Another important consideration is how well the program makes everything work together, including working with resources that exist outside the program.
Internal Resource Integration – Is keylinking/hotlinking available to quickly bring up appropriate support or reference resources by double-clicking on a key word? Are these links user-definable? Do context menus (e.g. right-click) provide immediate access to a complete set of study tools, e.g. in-program or web searches, the annotation system, copy options, pronunciation modules, language helps or other tools, xrefs or other topical content? Are they available as appropriate in all window types? Are pop-ups available where appropriate, and can they be suppressed?
Are wizards available to quickly identify and display library content related to a passage? Does this include user-created content? How easy is it to send Bible references to a verse list, and how much control does the user have over where the refs will appear in the list (e.g. which section)? Are “harmony” tools available to provide easy comparison of topically-paralleled sections (e.g Gospel parallels, Kings/Chronicles)? What other productivity tools are there?
Is there an internal text editor for creating documents, and how feature-rich is it? Does it have a learning spell checker? Does it support drag & drop editing, or creating tables, outlines, and graphical content? Does the program parse Biblical references in user documents to provide hyperlinks to support xref indexing and/or verse pop-up windows?
Integrating External Resources – Does the program support integrated web browsing? Are specific web sites integrated for their content or functionality? Can the user edit and maintain the list of integrated web sites? Are RSS or Atom feeds supported?
Does the program publish an API for acting as a server of resources or functions? Can the program utilize the functionality of other Bible Study programs? Can the program integrate the functionality of document-focused programs, such as Microsoft Office, OpenOffice.org, etc? Can audio Bibles be integrated?
Can documents created outside the program be integrated into the program? Does the program support standard document types, e.g. html/xml, pdf, rtf, doc, OO.o, mp3, wav, jpg, gif, png? Are their tagging capabilities leveraged by the program’s searching or xref tools? Are documents created within the program easily usable outside the program? What data format are they in? Does the program include stand-alone tools for supporting the use of external programs?
Working with Non-Biblical Resources – Can you search books using Boolean logic? Can you search using regular expressions? Can you search by Biblical reference? Are books tagged for topical searching? Is there a topical index? Can search results be saved? Are different search terms differentiated in results display (e.g. by different color highlighting)? Can the context of a search result be previewed? Is the search term(s) clearly identified in the book view (as opposed to in a results page)? Can you search within a page (simple find)?
How easy are books to navigate? Is a navigation history maintained? Can academic citations be generated? Can the user copy text with or without citations? Do books have useful Tables of Contents? Do books have useful title pages? Is there a tagging system for the user to identify topical or xref relevant passages? Is there a facility for annotating books? How are publisher footnotes/endnotes presented? Is text markup supported?
Usability and Presentation
Linking & Syncing – Do like books sync to each other? Can books be set to open synced? Can sync be turned off? Are there multiple sync channels, or a single, global context? Can the user determine which resources will drive the sync context and which will be passive? How does the program deal with syncing passages when verse numbering differences occur between translations?
Can the user pursue and access xrefs without disrupting the primary study context? Can books sync on values other than BCV (book, chapter, verse) context (e.g. original language word or Strong’s #, topical keyword, etc)?. Will setting the sync flag on a book change the book’s context or the sync context? Is the syncing mechanism easy to understand? Does it make studying easier, or does it get in the way?
Window Management – Can resource windows be docked into window sets? Are there limitations to what kinds of windows can be docked? Can window types be mixed within a dock set? Can data windows (e.g. documents, verse lists) be docked? Can docking be set to automatic? Can windows be moved within or between docked sets?
Is auto-arranging of windows available? Do auto-arrange window placements make sense? Is there any user control over placement or layout? Can the user create custom (non-auto) window layouts? Do newly opened window sizing and placement make sense? Can window titles (captions) be customized for easy identification?
Do target windows for viewing or following xrefs make sense, both in terms of the window chosen, and in terms of identifying the desired content within the target window? Can the user designate target windows for different purposes? Are window/context navigation tools easy to use and understand? Can multiple copies of particular resource windows be opened?
Saving Desktop Context & Layout – Can desktops be saved and retrieved? Can the window layout and study context(s) be saved and retrieved independently from each other? How much granularity does the user have in determining what gets saved with a desktop? Do desktops include desktop-specific data objects (e.g. desktop notes)?
Library Management – How easy are books and other resources to find? How clean is the library layout? How much control does the user have in the organization of the library layout? Are there useful tools for searching for books and other resources? Can the user hide little-used books, or remove unwanted ones? How easy is it to find, purchase, and add new books? How much information is available to the user regarding books being considered for purchase? Can you try before you buy with books? How easy is it to backup, restore, and manage licenses? Is a bibliographer available?
Usability of Interface – How well laid out is the application on the whole? Does it look cluttered? Is it easy to find resources? Is it easy to access functions? Is the menu system sensible? Is the icon bar or ribbon well designed? Do things behave the way you’d expect a Windows app to behave? How easy is it to accomplish typical tasks?
Are there sufficient keyboard shortcuts? How easy is it to hide, find, and keep track of open windows? How readable are the various windows? How much control does the user have over font size and selection? Are there options available for selecting from multiple color schemes, or creating custom ones?
Does the program utilize bookmarking? Does it provide a set of easily accessible “favorites” links? Does it support the saving of user-defined collections of resources for easy recall of common search scopes? Does it support the customization of defaults and orderings for various selection screens, file management functions, and other program behaviors?
Extra-biblical Library Resources
Books and other resources that provide interpretations and applications of God’s Word.
Evaluate the scope and quality of the catalog of available electronic books (including third-party content providers), such as: outlines, commentaries, handbooks, study bible notes, lesson plans, Bible studies, histories & surveys, theological works, doctrinal works, user community documents and books, sermon helps, illustrations, spiritual works, devotionals, how-to books, comparative literature, sacred art reproductions, hagiographies, hymns and other music, presentation graphics, etc.
Catholic Resources – What is available from the vast world of Catholic intellectual and spiritual works? For better or worse, the New American Bible is a must-have resource for American Catholics – at least those preparing anything for a parish setting – is the NAB available? Are other Catholic translations available (Douay, Knox, JB, NJB)? Are the Catholic editions of the RSV or NRSV available? Are other complete translations available with the disputed books included (KJV, RSV, NRSV, TEV, CEV, NEB, REB, LXX translations)?
Is the Catechism of the Catholic Church available? Are other formal doctrinal works available, e.g. conciliar and papal documents? Are any of the published doctrinal compendiums or surveys available? Are the typically extensive footnotes and Biblical xrefs in these works linked? Is the Summa Theologiae available? What about other Catholic theological works? Are the writings of the Early Church Fathers available?
What about Catholic spiritual works? What about Catholic Biblical commentaries and dictionaries? Is the Liturgy of the Hours (or even the daily Mass readings) available as a daily “devotional”?
Technical Foundation
Which OS platform(s) does the program run on? What is the format of the books? Is it proprietary or open source? What is the format of text copied from books? How well are the books designed and produced? How are program updates and patches distributed? Does the application platform appear to have a solid future?
How good are the help files? Is there other help available online, and in what format? Is there an online user community, and is it helpful? Is there training available? Is it free? Is there on-line or email tech support provided by the vendor? Telephone support? How good is the support? Is there an effective customer feedback process?
How is the overall performance of the application? How easy is it to install, update, backup? How fast does it startup? How fast does it perform tasks? Does the program behave? How buggy is it? Is it prone to crashing and errors? Does it handle multiple threads well, or does a slow process cause the whole app to stop performing? Do all the buttons work properly? Is the program functionally polished, or does it seem half finished?
Marketing and Pricing
What is the entry price for the product? What do you get for that price in terms of tools, functionality, and library resources. What is the cost for 100% of the functionality? What is the real cost to get to the level that the product markets itself at? What is the real cost to get to the level of your own base requirements? How is the pricing for individual add-on resources, compared to other publishers?
Is the pricing simple and clear enough that you know you’re getting the best deal when you buy, or does it feel like you’re probably paying more than you should? How easy is it to buy new resources? Are the available resources (and packages) explained well in the marketing material? Are there good examples? Is there user feedback/reviews? Can you try before you buy? Are there incentive programs? Is there price protection? Does the vendor seem solid, focused, reliable, and in it for the long haul?