Generative AI at UVA

Generative AI isn't limited to just text. It can produce images, videos, music, and other forms of media. From deepfake videos, which we will include in this category, to AI-generated artwork and music, the use and misuse of this technology in media is growing. In this guide, we’ll explore the types of generative non-text AI, examples of tools you can try, concerns and challenges, recent news, and resources to learn more.

Deepfakes: Imagine a video of a celebrity or politician saying something they never really did. Deepfakes are tools that can edit videos to make it look like someone is doing or saying something they haven't. Think of it like Photoshop but for videos. 

• DeepFaceLab (open source) https://www.deepfakevfx.com/downloads/deepfacelab/  
• DeepSwap www.deepswap.net/  
• ReFace https://reface.ai/  

Text to Image: If you've ever wished for a tool that could turn your words into photographs, illustrations, or digital paintings, this is it. Describe an image in words, and these tools try to create an image that matches. 

• Dall-E https://openai.com/dall-e-2/ 
• Midjourney https://midjourney.com/  
• Stable Diffusion https://stablediffusionweb.com/  
• Dream Studio https://beta.dreamstudio.ai/  
• ClipDrop https://clipdrop.co/  
• Deepai https://deepai.org/machine-learning-model/text2img  
• Craiyon https://www.craiyon.com/ 

Generative Video: Write a short description, or upload a photo, and generative video tools will aim to produce a video clip based on your description or image. 

• Runway (Text to Video, Image/Video to Video) https://runwayml.com/  
• ModelScope (Text To Video Synthesis) https://huggingface.co/spaces/damo-vilab/modelscope-text-to-video-synthesis  

3D/Animation: Describe an object or a character, or upload a photo, and these tools will craft it into a 3D model. Or make a 3D model move based on your movements. 

• Fantasia3D (Text to 3D model) https://fantasia3d.github.io/  
• Deep Motion (Motion tracking character animation) https://www.deepmotion.com/  
• Image to 3D model (examples ICON, PaMIR, pifuhd, MonoPort) 

Text to Music / Song Composition: Describe a musical composition in words, such as a mood or a theme, and these tools will turn your descriptions into music. 

• Boomy https://boomy.com/  
• AIVA https://www.aiva.ai/  
• OpenAI MuseNet https://openai.com/research/musenet  

Voice reproduction (text to speech, speech to speech): Enter some text, or your own recording, and these tools attempt to impersonate someone else. 

• Resemble AI https://www.resemble.ai/  
• Speechify www.speechify.com/ 
• Voice.ai www.voice.ai  

Vocal and Instrument Impersonation: These can accurately replicate specific voices or the sound of musical instruments. 

• Vocaloid https://www.vocaloid.com/en/ 

Other: there are many other exciting tools out there. 

• Tome (create slide decks, generative story-telling) https://beta.tome.app/   
• Image upscaling / Increase image resolution tools 

Tools and Resources for AI Art (List): https://pharmapsychotic.com/tools.html 

[Image from NYTimes Article: Pavich, Frank. “Opinion | This Film Does Not Exist.” The New York Times, 13 Jan. 2023, https://www.nytimes.com/interactive/2023/01/13/opinion/jodorowsky-dune-ai-tron.html. Accessed 22 Aug. 2023.] 

As fascinating as generative AI and deepfake technologies are, they come with a set of concerns. For the average person, these concerns could affect everything from how we consume news to how we trust online content. Here's a closer look at some of the challenges posed by these technologies and potential harm that we all could face: 

• How Fast This Is All Changing: The sheer speed at which generative AI progresses is daunting. This guide is a start, and will be updated as is possible, but staying up to date is a challenge. 

• Potential for Misinformation and Disinformation: The ability of AI to create compelling media might lead to widespread misinformation (spreading false information without intent) and disinformation (deliberately spreading false information), which can spread like wildfire.  

• Global Political Concerns: The potential to create fabricated videos or audio clips of world leaders can influence public opinion or spark international incidents.  

• Fake News: A direct offshoot of misinformation, AI-generated news, if unchecked, could lead to the propagation of biased or utterly false narratives. With the ability to generate believable fake news stories, people might be misled into forming opinions or making decisions based on falsehoods. 

• Potential for Personal Harm: Deepfakes can be used maliciously to impersonate individuals, leading to personal and professional damages. Personal videos or images could be manipulated to portray individuals in situations they were never in, leading to cyberbullying or reputational damage. 

• Deepfakes, Misuse, and Criminal Conduct: Deepfakes could be used for blackmail, fraud, or to create false evidence. For instance, students could be wrongly accused of misconduct through fabricated videos. 

• Criminal Justice Concerns: In legal settings, the reliability of evidence could be questioned if deepfakes or generated photographs become prevalent. The sophistication of fakes raises significant concerns in the justice system, especially when AI-generated content is presented as evidence. 

• Vocal Reproduction and Scams: Imagine getting a call from a 'family member' asking for emergency funds or personal details, only it's not them but an AI-generated voice. AI's ability to reproduce voices can be weaponized for scams, defaming individuals, or impersonating influential personalities. 

• Copyright and legal considerations – see tab entitled: “Copyright and Intellectual Property” 

• Kolirin, Lianne. “Artist Rejects Photo Prize after AI-Generated Image Wins Award.” CNN, 18 Apr. 2023, https://www.cnn.com/style/article/ai-photo-win-sony-scli-intl/index.html. 

• “AI Image Generator Fabricates ‘Archival’ Bigfoot Photo.” RumorGuard by News Literacy Project, 7 July 2023, https://rumorguard.org/post/ai-image-generator-fabricates-archival-bigfoot-photo. 

• Allyn, Bobby. “Deepfake Video of Zelenskyy Could Be ‘tip of the Iceberg’ in Info War, Experts Warn.” NPR, 16 Mar. 2022, https://www.npr.org/2022/03/16/1087062648/deepfake-video-zelenskyy-experts-war-manipulation-ukraine-russia. 

• Coscarelli, Joe. “An A.I. Hit of Fake ‘Drake’ and ‘The Weeknd’ Rattles the Music World.” The New York Times, 19 Apr. 2023, https://www.nytimes.com/2023/04/19/arts/music/ai-drake-the-weeknd-fake.html?smid=url-share. 

• “AI Rapper FN Meka Dropped by Capitol over Racial Stereotyping.” BBC News, 24 Aug. 2022, https://www.bbc.com/news/newsbeat-62659741.  

• Genovese, Daniella. “Spotify Removes AI-Generated Songs from Platform.” FOXBusiness, 10 May 2023, https://www.foxbusiness.com/lifestyle/spotify-removes-ai-generated-songs-platform.  

• “Amazon Music Strikes Playlist Partnership with Generative AI Music Company Endel.” Music Business Worldwide, 20 Feb. 2023, https://www.musicbusinessworldwide.com/amazon-music-strikes-playlist-partnership-with-generative-ai-music-company-endel12/. 

• Hsu, Tiffany. “Worries Grow That TikTok Is New Home for Manipulated Video and Photos.” The New York Times, 4 Nov. 2022, https://www.nytimes.com/2022/11/04/technology/tiktok-deepfakes-disinformation.html?smid=url-share. 

• Verma, Pranshu. “They Thought Loved Ones Were Calling for Help. It Was an AI Scam.” Washington Post, 10 Mar. 2023, https://www.washingtonpost.com/technology/2023/03/05/ai-voice-scam/. 

• Greene, Casey, et al. “AI and the Future of Work: 5 Experts on What ChatGPT, DALL-E and Other AI Tools Mean for Artists and Knowledge Workers.” The Conversation, 11 Jan. 2023, http://theconversation.com/ai-and-the-future-of-work-5-experts-on-what-chatgpt-dall-e-and-other-ai-tools-mean-for-artists-and-knowledge-workers-196783. 

• Marr, Bernard. “Deepfakes – The Good, The Bad, And The Ugly.” Forbes, https://www.forbes.com/sites/bernardmarr/2022/01/11/deepfakes--the-good-the-bad-and-the-ugly/. 

• Pavich, Frank. “Opinion | This Film Does Not Exist.” The New York Times, 13 Jan. 2023, https://www.nytimes.com/interactive/2023/01/13/opinion/jodorowsky-dune-ai-tron.html. 

• “Jodorowsky’s Tron | Cobra Blade Detective Agency -- Johnny Darrell.” Mysite, https://www.johnnydarrell.com/jodorowskystron. 

• Sam Finn – A.I. Photography (Contemporary Artist Using Generative A.I.): “AI.S.A.M.” AI.S.A.M, https://www.aisam.xyz. 

• No, Tom Cruise Isn’t on TikTok. It’s a Deepfake | CNN Business. 2021, https://www.cnn.com/videos/business/2021/03/02/tom-cruise-tiktok-deepfake-orig.cnn-business. 

• Robinson, Tasha. “Wes Anderson AI Art Is a Fun Way to Clarify AI’s Ethical Issues.” Polygon, 7 Dec. 2022, https://www.polygon.com/23494958/wes-anderson-midjourney-ai-art-generator-viral-trend. 

• Thorud, Josh. "Explore the World of AI Media and Non-Textual Tools." UVA Center for Teaching Excellence Teaching Hub. https://teaching.virginia.edu/collections/josh-thoruds-generative-ai-collection

• RumorGuard from the News Literacy Project: https://rumorguard.org/ 

• “Can AI and Creativity Coexist?” UCLA, https://newsroom.ucla.edu/stories/artificial-ingelligence-and-creativity-coexist-foster-snelson. Accessed 22 Aug. 2023.  

• Dai, Yun. Research Guides: Machines and Society: Image Generation Tools: https://guides.nyu.edu/data/ai-image-generation 

• “Student Guide for AI Use.” AI for Education, https://www.aiforeducation.io/ai-resources/student-guide-ai-use. Accessed 22 Aug. 2023. 

• “10 Best AI Voice Generators (August 2023)” Unite.AI. https://www.unite.ai/10-best-ai-voice-generators/.  

• Bartels, Meghan. “How to Tell If a Photo Is an AI-Generated Fake.” Scientific American, https://www.scientificamerican.com/article/how-to-tell-if-a-photo-is-an-ai-generated-fake/. 

• Groh, Matthew. “10 Ways to Spot Fake Videos and Falsehoods on the Internet.” MIT Media Lab, https://www.media.mit.edu/articles/10-ways-to-spot-fake-videos-and-falsehoods-on-the-internet/.  

• Palamara, Jason. “3 Ways AI Is Transforming Music.” The Conversation, 14 Aug. 2023, http://theconversation.com/3-ways-ai-is-transforming-music-210598. 

• Tutorial: “Lateral Reading.” Checkology by News Literacy Project. https://checkology.org/demo/lesson/3a56e60272f549d76b0254bd008268faec4adfd2  

Generative A.I. Images and Media Zotero Library: 

https://www.zotero.org/groups/5154226/uva_library_ai_media