This post was done in about 10 minutes…so consider it a conversation starter that needs input.
I had a very interesting conversation talking about project ideas for AI and academic libraries. It quickly focused on AskA Librarian services and good ‘ol digital reference/virtual reference. Imagine, the conversation went, we could take a person’s question and run it through chatGPT and then prompt librarians to work with the person with the AI prompt.
I jumped back about 30 years to discussions of “sandwich interfaces” that would search against a database of previous questions and answers. Then we could look for different types of questions asked, and which would work better with AI. Could we remake QuestionPoint (now LibAnswers https://springshare.com/libanswers/ ) in a librarians driven augmented intelligence function. One that could drive collection development and interface design? Take the virtual reference work o Conway and Radford and join it with Soo Young Rieh’s work in learning and search interfaces!
This then dove into previous studies, like Joe Janes’ work on how many times librarians use reference interviews online, McClure’s Correct Answer fill rate (and the infamous 55% rule), Makiko Maiwa’s research into reference questions asked at different points in an information literacy process, even the work the MacArthur Foundation funded on Reference Extract and the use of Q&A archives to develop credibility scores to integrate into search engines.
It was a sudden moment that showed that a lot of the virtual reference work of the 90’s and early “aughts” could be a very useful framework for the integration of generative AI into reference work. Imagine training large language models on reference data (deidentified). Imagine feeding this type of high value data into generative process to speed the development of research documentation, and improve search interfaces on large databases.
Don’t like chatGPT, we could use open AI and large language model software to dig into the depths of all that human expert labor in Q&A and see what emerges.
I made a joke about throwing chatGPT into random spots of my old papers and resubmitting them. Seriously, a first take at new titles for old papers:
[means of evaluating chatGPT output and prompt engineering] Kasowitz, A., Bennett, B., & Lankes, R. David (2000). Quality standards for digital reference consortia. Reference and User Services Quarterly, 39(4), 355-363.
[determining the utility of responsiveness in conversational modes of AI interactions] Lankes, R. David & Shostack, P. (2002). The necessity of real-time: Fact and fiction in digital reference systems. Reference and User Services Quarterly, 41(4), 350-355.
[Identifying attributes and structures in generative AI prompts] Belanger, Y.M., Lankes, R. David, & Shostack, P.L. (2003). Managing data collection for real time reference: Lessons from the AskERIC Live! experience. Internet Reference Services Quarterly, 8(1/2), 137-149.
[applicability of human mediated Q&A systems for ReferenceAI] Pomerantz, J., Nicholson, S., & Lankes, R. David (2003). Digital reference triage: An investigation using the Delphi Method into the factors influencing question routing and assignment. The Library Quarterly, 73(2), 103-120.
[utility of reference databases of questions and answers for the training of large language models in AI] Nicholson, S. & Lankes, R. David (2007). The Digital Reference Electronic Warehouse (DREW) project: Creating the infrastructure for digital reference research through a multi-disciplinary knowledge base. Reference and User Services Quarterly, 46(3), 45-59.
[transitioning human mediated systems to hybrid human/AI systems for credible reference] Lankes, R. David (2008). Collecting conversations in a massive scale world. Library Resources & Technical Services, 52(2), 12-18.
[just keep the same title] Lankes , R. David (1998). The virtual reference desk: Building human expertise into information systems. In Preston, C. (Ed.), 1998 proceedings of the 61st annual meeting (vol. 35). (pp. 81-90). Silver Spring, MD: The American Society for Information Science.
[same title] Pomerantz, J., & Lankes, R. David (2003). Taxonomies for automated question triage in digital reference. In C. C. Marshall, G. Henry & L. Delcambre (Eds.), Proceedings of the 2003 joint conference on digital libraries, May 27-31 2003, Rice University, Houston, TX. (pp. 119-121). Washington DC: IEEE Computer Society.
Reinvigorate Reference Extract?
https://vimeo.com/manage/videos/80908278
https://vimeo.com/manage/videos/3335162
Ah man, this could be so fun. At the very least it is the beginning of a better narrative than “libraries are doomed,” “reference is dead,” etc.
yep – worth a try to determine relevance and credibility. CRED is the key. Start w AI but focus on credibility. Bing Chat provides references. So does ChatGPT 4, I’m told. Make it better.
As for undergrads and turning in research papers done by ChatGPT, flip the assignment. Require a ChatGPT generated paper BUT the actual assignment is to analyze the generated paper for credibility – accuracy, reliability currency.
Let’s talk about this and more in the next Libraries Lead podcast – https://librarieslead.buzzsprout.com/
I’ll be listening! So far for me, the best evaluation is with a known topic.
I used cancer, which I’ve been researching, and have read about all too much. A query about leukemia & multiple myeloma in Bard gave me a very good summary (sans references) of what I know is up-to-date information, and when asked, a brief list of resources – all top-level from federal health agencies, none from GScholar. None inaccurate, either, and both cancers are experiencing a high level of research and treatment changes. (Query: adult 65+ diagnosed with b-all and multiple myeloma: prognosis and treatment.)
From there, students could investigate claims using PubMed, etc.
Correction – that was ChatGPT, not Bard.
In a few more years we will have computing power and AI that can read simultaneously all the content in the Library of Congress. Thank you for this fun piece and ruminations! 🙂