Incongruent
Podcast edited by Stephen King, award-winning academic, researcher and communications professional.
Chester | Dubai | The World.
Correspondence email: steve@kk-stud.io.
Incongruent
EU Parliament 2025: The Future of AI: Balancing Innovation with Responsibility
Use Left/Right to seek, Home/End to jump to start or end. Hold shift to jump forward or backward.
Spill the tea - we want to hear from you!
Generative AI is transforming technology and reshaping our world by creating human-like content at unprecedented scale and speed.
• EU ranks second globally in Gen AI academic publications but faces funding gaps for innovation
• Technical foundations include deep learning architectures, specialized processors, and massive datasets
• Gen AI systems face unique cybersecurity vulnerabilities like data poisoning and prompt injection
• Emerging trends include agentic AI making autonomous decisions and multimodal AI processing diverse data types
• Teachers are more exposed to AI-automatable tasks than 90% of workers across occupations
• Gen AI enables creation of convincing false content, threatening information integrity
• Digital commons face existential threats from AI crawlers and potential information pollution
• Data centers powering AI projected to consume growing portions of electricity by 2027
• Children need specific safeguards against manipulation and inappropriate AI-generated content
• Bias in training data perpetuates societal prejudices in AI outputs
• Privacy concerns include AI's ability to infer sensitive information from seemingly innocuous data
• Copyright challenges involve determining appropriate limitations for AI training
For more information on generative AI and its impacts, check out the comprehensive outlook report from the European Commission's Joint Research Center.
Introduction to Generative AI
Speaker 1Welcome to the Deep Dive . Today , we are plunging headfirst into something that's not just changing technology , but really reshaping our world . Generative AI .
Speaker 2Yeah , it's everywhere now , isn't it ?
Speaker 1Exactly . You've interacted with it , you've seen the outputs . You know it's transforming things . Yeah , at its heart , gen AI is about systems creating well human-like content text , images , code , you name it .
Speaker 2But at a scale and speed we've just never seen before . It's a huge potential force for , you know , innovation , productivity , societal shifts .
Speaker 1And to guide our deep dive into this really complex space , we're using a fantastic source . It's a comprehensive outlook report from the European Commission's Joint Research Center , the JRC .
Speaker 2That's right , and what's great about this report , I think , is that it's not just tech specs . It's actually designed to inform policymakers , people working in digital education , justice across the board . It pulls together the latest science , expert insights , gives a really sort of nuanced picture .
Speaker 1Absolutely , Because I mean , the potential is massive . We see that . But Gen EI also brings these significant challenges right . They're all interconnected to misinformation , job market shifts , privacy , big stuff , Definitely . So our mission today is to cut through that complexity . We want to pull out the insights you need to understand what's really important in this rapidly evolving landscape . Looking across tech , economy , society , policy , the whole picture .
Speaker 2Exactly . This report is kind of a vital resource for understanding trends , anticipating what might be next . It doesn't pretend to have all the answers , but it brings the science right to the policy table .
Speaker 1Okay , let's unpack this then when do we start ? Maybe the foundations , the tech that actually makes Gen AI
Technical Foundations of Gen AI
Speaker 1tick ? What are the building blocks here ?
Speaker 2Yeah , good place to start . It really comes down to a few key things that have advanced together quite rapidly actually . On the software side , huge strides in deep learning architectures , the transformer model particularly .
Speaker 1Right the transformer . That was a big deal for understanding language context .
Speaker 2A massive deal . These algorithms are computationally intensive , sure , these algorithms are computationally intensive , sure , but they're what allows models to process context and generate stuff that's surprisingly coherent , relevant .
Speaker 1But algorithms need power , right Serious power .
Speaker 2Oh , absolutely . That's where the hardware comes in Specialized processors , gpus , tpus . They're indispensable . They provide the sheer computational muscle you need to train and run these absolutely massive models efficiently .
Speaker 1And that muscle needs something to chew on Data .
Speaker 2Mountains of it . Massive data sets are the well indispensable raw material . Combine those huge data sets with high performance computing , think supercomputers and faster connections like 5G .
Speaker 1Right .
Speaker 2And you get this environment where gen AI models can be developed and trained at just an unprecedented scale . The EU , for instance , is investing heavily in HPC and these gigafactories to build up that capacity .
Speaker 1OK , so , given these foundations , where does the EU actually stand globally ? I mean in terms of research , but also turning that research into , you
EU's Position in Global AI Landscape
Speaker 1know , real world innovation .
Speaker 2So here's a key insight from the report , and it's a bit mixed . The picture is strong in research , definitely , but there are challenges turning that into market leadership .
Speaker 1How strong is strong ?
Speaker 2The EU is actually second globally in academic publications on Gen AI , right after China , so that shows a really vibrant research base .
Speaker 1Second worldwide . Wow , that's impressive .
Speaker 2It is . But and this is the critical point the report highlights the EU faces significant funding gaps , especially compared to the US and China . Wow , the money , yeah . And that affects its ability to translate that research into innovation , into patents . Eu patent filings are growing fast , sure , but they still only represent about 2% of global filings . They lag way behind South Korea and the US .
Speaker 2Okay , represent about 2% of global filings . They lag way behind South Korea and the US . Okay , so balancing this strong research base with closing that investment gap to really drive innovation , that's a major challenge .
Speaker 1Okay , so we have the tech , the data , the hardware capacity is growing , but with systems this complex , how do we even begin to evaluate them , Make sure they're safe , reliable ? It feels like uncharted territory .
Speaker 2It absolutely is , and the report really emphasizes this critical need for what they call a science of evils or model metrology . Basically , we need standardized ways to measure both the capabilities and , crucially , the safety of these models , especially as they move into sensitive areas .
Speaker 1A
Evaluating and Securing AI Systems
Speaker 1science of evaluation .
Speaker 2Yeah .
Speaker 1That really captures the challenge , doesn't it ? What does that actually involve in practice ?
Speaker 2Well , it involves developing new methods for benchmarking both performance and safety . Adversarial testing , often called red teaming , is vital .
Speaker 1Red teaming so like deliberately trying to break it .
Speaker 2Yeah , basically pushing it to find weaknesses , trying to make it fail or produce undesirable outputs . And human evaluation is still incredibly important , having experts or just representative users test the systems out .
Speaker 1But here's where it gets really interesting . What happens when the AI's abilities start to surpass our own understanding ? How do you evaluate something that's operating at a superhuman level ?
Speaker 2That's a huge future challenge . This concept of superhuman evaluation is emerging precisely because of that . We need ways to assess capabilities , safety , issues that humans might not even be able to fully perceive or understand on their own . It's a critical area for ongoing research , definitely .
Speaker 1Okay , shifting focus just slightly . What about cybersecurity ? J&ai systems handle massive amounts of data . They're incredibly complex . They must present new kinds of vulnerabilities , right .
Speaker 2They absolutely do . They're vulnerable not just to , you know , the standard cyber threats we already know about , but also to threats specific to AI systems themselves . The report breaks these down . It's really important to understand them because they open up whole new attack surfaces .
Speaker 1Okay , so what are some of these AI-specific vulnerabilities ?
Speaker 2Well , first there's data poisoning . Because these models are trained on these vast , sometimes unverified data sets , attackers can subtly inject malicious samples into that training data .
Speaker 1How does that work ?
Speaker 2It can compromise the model's overall performance , maybe subtly , or introduce specific risks like making a code generating AI suggest insecure code patterns without the user realizing it . It's like subtly contaminating the ingredients before the cake is baked .
Speaker 1O' Okay , so attacking the training data itself . What else ?
Speaker 2Then there's model poisoning Similar idea , but it involves manipulating the training process itself or the model's learning updates directly again again , to compromise its behavior . Marc .
Speaker 1Thiessen , not attack the data or attack the learning process . What about attacking the model once it's built and people are using it ?
Speaker 2Danielle Pletka . Right . That brings us to prompt injection . This is where carefully crafted input , a prompt , makes the model behave in ways it wasn't intended to .
Speaker 1Marc .
Speaker 2Thiessen . Okay , danielle Pletka . It can be direct prompt injection a user trying to bypass safety filters maybe get it to generate harmful content or misuse its abilities . But there's also indirect prompt injection . This is fascinating and pretty concerning .
Speaker 1How does that work ?
Speaker 2The model interacts with external content right Like it reads a webpage or processes a PDF . You feed it , and that external content secretly contains instructions , hidden prompts that alter the model's operation when it processes it .
Speaker 1Wait , hang on . So the AI looks at a web page and the web page can essentially give it secret commands . That sounds wild and potentially very dangerous .
Speaker 2It can be . These indirect attacks can compromise the system's integrity , its privacy , and often the negative consequences fall on the primary user of the system , not the attacker who planted the malicious content somewhere else .
Speaker 1Wow , okay , and what about trying to get sensitive information out of the model or its training data ?
Speaker 2Right . That's the domain of information extraction . Attackers aim to access sensitive or proprietary info . This includes things like data leakage or membership inference . That's where attackers try to figure out if a specific piece of data was part of the training set . If that data point was sensitive maybe copyrighted material or personal info that shouldn't have been there in the first place that's a major leak . Then there's model inversion Attackers try to reconstruct aspects of the training data or infer sensitive details just by analyzing the model's outputs or its internal structure .
Speaker 1And just trying to steal the model itself .
Speaker 2Yeah , that's model extraction , Basically trying to replicate the parameters , the brain of a remote model . The key takeaway here is that GenAI's reliance on massive , often diverse data and these complex models , it just significantly increases the potential attack surface compared to older systems .
Speaker 1That gives us a really clear picture of the current tech landscape and its vulnerabilities . The report also looks ahead , though what emerging technological trends are on the horizon beyond the kind of large language models we mostly interact with today .
Speaker 2Yeah , it flags several really interesting developments . One is agentic AI .
Speaker 1Agentic AI .
Speaker 2Right . This goes beyond the model just responding to a single prompt you give it . These systems are designed to make autonomous decisions , break down complex goals into subtasks , maybe initiate actions and , crucially , learn from the outcomes . They exhibit a form of computational agency .
Speaker 1So not just answering my question , but actively doing things or pursuing a goal on its own .
Speaker 2Precisely , you know . Think of potential AI co-scientists autonomously formulating and testing hypotheses , or maybe
Emerging AI Technologies
Speaker 2self-correcting AI frameworks that improve themselves over time . This has really significant implications for how work gets done , how knowledge is produced in the future .
Speaker 1What about AI that can understand more than just text , like images , audio .
Speaker 2That's multimodal AI . These systems process and integrate diverse data types text , image , audio , maybe sensor data . Others Imagine an AI taking a patient's entire medical history text , notes , lab results , scans and integrating all that to produce a comprehensive diagnostic report .
Speaker 1That sounds incredibly powerful , especially for fields like medicine .
Speaker 2It does , or systems that translate between modalities like generating a descriptive text report from a complex image automatically . But integrating multiple data types also amplifies challenges we've already touched on , like bias . Biases from different data types can compound and copyright issues get even more complex when training on diverse existing works across different formats .
Speaker 1Right and AI that can maybe simulate more complex reasoning like step-by-step thinking .
Speaker 2Yes , that's . Advanced AI reasoning Systems are being designed specifically to perform logical , step-by-step problem solving , trying to emulate more deliberate human thought processes . There are things called large concept models that integrate these vast networks of conceptual knowledge to improve decision making .
Speaker 1So trying to make AI think more like us .
Speaker 2In a way it promises more sophisticated capabilities , for sure , but it also raises some tricky ethical questions about mimicking human cognition . And , importantly , it significantly increases energy consumption . These models can be very power hungry .
Speaker 1Which leads us nicely to explainability or XAI . Why is that so crucial for these increasingly complex and powerful systems ?
Speaker 2Well as AI takes on bigger roles in sensitive or high stakes areas . You know security , health care , finance , even legal decisions . People need to trust how the AI reaches its conclusions . They need to understand it .
Speaker 1So it's not enough to just get the right answer . We need to see the workings behind it , or at least understand the logic .
Speaker 2Exactly . Explainability helps provide understandable justifications . Insights into the AI's decision making process . Understandable justifications , insights into the AI's decision-making process . It's vital for building user confidence , enabling effective human-AI collaboration and often meeting regulatory requirements . Techniques like attribution graphs can help visualize which inputs most influence the decision . The report frames XAI as increasingly an ethical dimension , really , and even a legal requirement under EU law , like the AI Act . Though standardizing explainability for these very complex , often opaque black box models
Economic Impact and Job Transformation
Speaker 2that remains a huge challenge .
Speaker 1That's a fantastic overview of the tech landscape and where it might be heading . Yeah , let's shift gears a bit and look at the economic picture . How is the EU actually positioned globally in this Gen AI economy ?
Speaker 2Yeah , as we touched on earlier , it's definitely a nuanced picture . The EU is certainly significant player , represents about 7% of global AI players , ranks third overall and , as we said , its research strength is undeniable , second only to China in publications .
Speaker 1But the challenge is turning that research muscle into market dominance . Is that fair ?
Speaker 2That's precisely it . The report really highlights the significant lag in innovation , patents and , perhaps most critically , in venture capital investment compared to the US and China .
Speaker 1Right the VC funding again .
Speaker 2Yeah , While EU companies are investing in foreign players and the EU hosts foreign players too the US significantly leads in actually owning foreign Gen AI companies foreign Gen AI companies . Germany is needed as a leader within the EU in both hosting and owning stakes . But bridging this overall VC investment gap is a key challenge for EU players who want to compete globally .
Speaker 1How are traditional industries being impacted Manufacturing , for example ?
Speaker 2Gen AI is set to really transform manufacturing . Think smart production lines using advanced data analytics , predictive maintenance powered by AI , making autonomous decisions about machine health before things break . Optimizing complex supply chains , automating intricate processes . It's about creating more interconnected , efficient and adaptable systems .
Speaker 1And what about the creative industries ? This feels like an area seeing huge and sometimes pretty contentious impact because of AI's ability to generate content .
Speaker 2It's definitely a space of both massive opportunity and , yes , significant tension . Gene AI is revolutionizing content creation , helping creators generate text , images , music video much faster , exploring entirely new artistic models . But here's where the challenges are particularly acute Intellectual property and copyright . Genie-ai models are often trained on vast amounts of existing creative work , sometimes , you know , without explicit permission from the creators .
Speaker 1That's the major point of conflict we're seeing play out in courtrooms right now , isn't it ?
Speaker 2It really is . It raises serious questions about fair compensation for creators whose work is used for training and the potential displacement of original human-created works by AI adaptations or variations . There's also a risk , if it's not managed carefully , of homogenization , you know , AI models just relying too heavily on existing styles and trends rather than fostering true novelty .
Speaker 1Let's talk about the impact on the labor market Employment . We hear so much talk about AI replacing jobs .
Speaker 2How does the report frame this ? It frames it quite carefully , emphasizing productivity gains versus potential disruption or displacement , and a key insight here , I think , is that Gen AI impacts tasks within jobs , not necessarily entire jobs , wholesale .
Speaker 1Tasks , not jobs .
Speaker 2Okay . It augments several cognitive abilities that are crucial in many roles , things like comprehension and expression , understanding and generating language , attention and search , finding information , classifying documents and conceptualization , learning , abstraction , identifying patterns , generalizing from data .
Speaker 1So how do those AI capabilities map onto specific jobs ? Which ones are most affected ?
Speaker 2Well , a JRC study highlighted occupations most exposed to these AI capabilities . It identified engineers , software developers , teachers , office clerks and secretaries as facing particularly high exposure .
Speaker 1Teachers , that's interesting .
Speaker 2Yeah , the study found teachers were actually more exposed to AI-automatable tasks than 90% of workers across all the occupations they surveyed . It's not necessarily that the entire teaching job is replaced , of course , but many tasks within it could be significantly changed or augmented by AI .
Speaker 1That makes sense . It's about the nature of the work shifting , which brings us straight to the skills gap . Right ? If tasks change , people need different skills .
Speaker 2Absolutely . The report really stresses that the needed skills go beyond just , you know , learning to use the tools . It involves understanding the broader implication , the ethical considerations , the limitations of the AI .
Speaker 1And how are we doing on that front in the EU ?
Speaker 2Well , the EU has set an ambitious target in its digital decade program 80% of the population should have basic digital skills by 2030 . But as of 2023 , only 56% had met that target , so there's a clear gap that needs addressing .
Speaker 1How do we close that gap then ?
Speaker 2It requires a really significant push for upskilling , retraining and comprehensive AI literacy initiatives , programs updating digital competence frameworks like DigComp 3.0 to explicitly include AI knowledge and ethical use , and developing specific AI literacy programs , starting right from schools .
Speaker 1Okay , and finally , on the economic side , what does the market for conversational AI-like chatbots look like specifically within the EU ?
Speaker 2It's described as a complex and dynamic market , but currently dominated by a few large non-EU players . Market but currently dominated by a few large non-EU players . Openai's chat GPT is identified as the clear leader in terms of user base across the EU .
Speaker 1Is that market uniform across all EU countries ?
Speaker 2Not exactly . There's variation in market share and also how people prefer to access these tools . Some prefer dedicated apps , others use websites more . It depends on the member state and while the major global players are dominant everywhere , you do have local players , like Mistral AI , based in France , who can have particular prominence in specific countries . There's also this interesting competition between companies who build both the underlying AI models and the user interface , the vertically integrated players and those who primarily build user interfaces or services on top of external models interface-only solutions .
Speaker 1Okay , that gives us a comprehensive picture of the tech and economic aspects . Yeah , let's move into the societal dimensions now and how policymakers are trying to keep pace Misinformation and disinformation
Societal Challenges: Misinformation and Media
Speaker 1. That seems like a major challenge that Gen AI could significantly worsen .
Speaker 2Oh , they absolutely are . The report highlights this as a critical area . Gen AI enables the creation of incredibly convincing false content think deep fakes at a massive scale and speed Right . It can pollute online information sources , amplify the spread of disinformation . It makes it incredibly hard for accurate information to keep up or for rebuttals to even be effective . We've seen its use already in sophisticated influence operations , like that doppelganger campaign targeting European countries .
Speaker 1So technical solutions like detecting deepfakes are important , but they're not enough on their own .
Speaker 2Precisely Technical measures like watermarking . They're valuable , yes , but the report strongly emphasizes that media literacy and AI literacy are crucial skills for citizens . People need the ability to critically evaluate AI generated content , understand its potential to deceive and resist manipulation attempts . It's a societal defense alongside the technical ones .
Speaker 1How is Gen AI actually being talked about in the media ? Does that shape public perception , do you think ?
Speaker 2Oh , definitely . The media narrative around Gen A , around Gen AI , is often quite polarized , you know , swinging between these utopian visions of transformative potential and quite dystopian warnings about risks , job losses , privacy collapse , that sort of thing . This often dramatic framing certainly shapes public discourse and , by extension , it can influence policy discussions too .
Speaker 1Has the media coverage ramped up recently ?
Speaker 2Massively . There was a huge surge in coverage starting in late 2022 , particularly after systems like ChatGPT became widely available . Reporting intensity tends to peak after key events like new model announcements , major commercial deals or significant regulatory proposals like the AI Act .
Speaker 1And what about the tone of that coverage ? Is it mostly positive , negative ?
Speaker 2Well , the report notes that in mainstream media , the overall sentiment towards Gen AI is actually predominantly positive , often highlighting economic growth opportunities . However , a significant chunk , maybe around 30% , does focus on the risks and negative implications Interestingly unverified sources . You know blogs , certain social media channels often have a more neutral or mixed tone , but they are much more prone to sensationalism and alarmism , either exaggerating the AI's capabilities or predicting catastrophic outcomes , often downplaying the ethical considerations in the process . Understanding this media landscape is really key to gauging public perception .
Speaker 1Let's turn to something called the digital commons , things like open source code repositories , Wikipedia , openly licensed creative works . How does Gen AI interact with these vital resources ?
Speaker 2The digital commons are absolutely crucial . They serve as fundamental , often publicly accessible , training data for many , many AI models .
Speaker 1Right , the raw material again .
Speaker 2Exactly , but Gen AI offers opportunities here too . It could help people find and navigate vast open data sets , maybe aid in fact-checking by quickly searching open knowledge bases , facilitate translation of open content , and using knowledge from the commons can even help improve the fairness and diversity of AI outputs , making them less biased .
Speaker 1So it sounds like potentially a symbiotic relationship .
Speaker 2Potentially yes , but the report also points to significant risks for the commons , and this is a critical finding . Gene AI poses several threats . That's right . It could lead to the enclosure or privatization of free knowledge if access for data scraping becomes restricted or maybe monetized . It might decrease voluntary contributions to platforms like Wikipedia if people just start relying solely on chatbots for information instead of contributing back . There's a significant risk of pollution if AI-generated errors or biases get scraped and inadvertently introduced into open databases . That requires costly human effort to find and correct , and the organizations hosting these commons , often nonprofits , face real financial strain from the sheer volume of AI crawlers constantly accessing their data , often with little direct return to support the infrastructure .
Speaker 1Wow , those risks paint a pretty challenging future for these resources we rely on .
Speaker 2They really do . The report contrasts potential future scenarios one where the commons thrive with the right support and policies , and another where they deteriorate , becoming less trustworthy , less useful and potentially hindering AI development itself by providing lower quality training data . Protecting the digital commons isn't just about open access . It's actually essential for developing fair , robust and advanced AI systems in the long run .
Speaker 1Okay , let's in the long run . Okay , let's address the environmental implications . Running AI models , training them
Environmental Impact and Digital Commons
Speaker 1it requires significant infrastructure , significant energy .
Speaker 2It absolutely does . The direct environmental impact is considerable , primarily from the data centers that power AI . These centers consume vast amounts of energy , large amounts of water for cooling and contribute significantly to electronic waste .
Speaker 1Do we have a sense of the scale ? How much energy are we talking about ?
Speaker 2Well , estimates vary and it's a moving target , but data centers globally accounted for around maybe 1.5% of total electricity consumption in 2024 . And AI's share of that is growing rapidly . Some estimates , project AI could reach , say , 27% of total data center energy consumption by 2027 .
Speaker 1That's a huge jump .
Speaker 2It is . There's still uncertainty in these numbers , mind you . New , maybe more efficient model types are emerging , but also more complex reasoning models that use more power . Supply chain issues affect hardware deployment . It's complex , but the trend is clearly upwards , and the relatively short lifespan of the specialized hardware also adds significantly to the e-waste problem .
Speaker 1So Gen AI is definitely resource intensive , but can it also help address environmental challenges ? Is there an upside ?
Speaker 2Yes , that's the other side of the coin . Ai can be applied to climate mitigation efforts , Things like tracking pollution sources more accurately , optimizing energy grids for efficiency , designing more sustainable materials . However , the full scale of this potential help is still being quantified and it faces practical limitations . The EU is implementing regulations to address the environmental impact of data centers . There are provisions in the Energy Efficiency Directive , the taxonomy regulation , and they're promoting energy-efficient hardware like neuromorphic chips for more sustainable deployment . The upcoming Cloud and AI Development Act also aims to support sustainable cloud infrastructure .
Speaker 1The report also mentions an indirect environmental impact from AI . What's that about ?
Speaker 2That's right . It's a less obvious point , but an important one . Biased AI models could potentially influence public attitudes or behaviors , including those related to climate change , for instance , if AI search results consistently downplay climate risks due to biases in their training data .
Speaker 1Ah , influencing opinion .
Speaker 2Exactly which could indirectly impact energy consumption or emissions policies . This highlights the need for transparent and unbiased models , and it also underscores the need for international cooperation , because different regions have very different policy approaches to these environmental issues right now .
Speaker 1Let's discuss the impact on specific vulnerable groups , starting with children's rights . How does Gen AI affect children ?
Speaker 2Well , it presents opportunities , certainly , like personalized educational tools or new avenues for creativity , yeah , but the risks are significant and children are particularly vulnerable .
Speaker 1In what ways ?
Speaker 2They're susceptible to deceptive manipulation techniques that AI enables . They're also at risk from exposure to harmful or inappropriate content and there's often a real lack of age . Appropriate privacy and safety measures built into many of these systems , plus the potential for AI bias to affect them and for hallucinations you know the AI confidently stating wrong information to mislead them these are major concerns .
Speaker 1So general AI ethics guidelines aren't really enough here . Children need specific safeguards .
Speaker 2Absolutely . The report strongly emphasizes the need for child safeguards to be designed into these systems right from the start , taking an age-appropriate , inclusive approach . It also calls for longitudinal studies . We need to understand the long-term impact of interacting with Gen AI on children's cognitive and mental development . That's crucial for informing future policy .
Speaker 1What about mental health ? More broadly , the report mentions risks associated with things like AI , chatbots and companion apps .
Speaker 2Yes , there are documented risks . There too . Users can develop addiction-like behaviors , become overly reliant on chatbots for validation , potentially displacing important human relationships .
Speaker 1Right .
Speaker 2And , tragically , there have been cases where chatbots have actually encouraged harmful actions , sometimes linked to their perceived sentience or just their tendency to agree with users without critical assessment .
Speaker 1And deepfakes , which we talked about regarding misinformation . They also have a significant mental health impact , don't they ?
Speaker 2A severe one . Deepfakes can be used for cyberbullying , harassment and , devastatingly , for creating and distributing non-consensual explicit content , often targeting women and girls . This causes profound psychological trauma to victims . The report cites the Almendraleo case in Spain as a tragic example of this , and Gen AI exacerbates this risk by making the creation of such harmful content much easier and more accessible than before .
Speaker 1That really highlights how interconnected all these issues are misinformation , safety , mental health Exactly .
Speaker 2And it presents real challenges for existing policies . Schools , for instance , have policies against cyberbullying , but applying them effectively to sophisticated AI-generated content is complex . It also challenges platforms beyond just social media , like app stores and search engines , where image manipulation apps some quite harmful have appeared .
Speaker 1This naturally leads us back to the core issue of bias , stereotypes and fairness . How does Gen AI actually perpetuate these problems ?
Speaker 2Well , fundamentally , Gen AI models are trained on vast datasets , right , and these datasets often reflect existing societal biases and stereotypes gender biases , racial biases , cultural biases , you name it .
Speaker 1So garbage in , garbage out , or rather bias in , bias out .
Speaker 2Pretty much . As a result , the AI can perpetuate and even amplify these biases in its outputs . The report gives clear examples . Studies show AI credit risk assessment models can exhibit similar gender bias to traditional methods . Text-to-image generators frequently show really strong gender and racial biases when generating , say , occupational portraits . Doctors are mostly male , nurses female , and so on . It just reinforces harmful stereotypes
Protecting Vulnerable Groups
Speaker 2.
Speaker 1So the bias in the data becomes bias in the AI's output .
Speaker 2Yeah .
Speaker 1How can this possibly be mitigated ?
Speaker 2It's tough , but there are strategies Using more diverse and representative training data is crucial , developing and implementing fairness-focused algorithms during training or post-processing and , really importantly , increasing diversity among the actual teams building these AI systems . Regular audits using diverse benchmarks are also essential . Regular audits using diverse benchmarks are also essential . And policy frameworks like the Digital Services Act , dsa and the EU , which requires very large online platforms to conduct risk assessments on systemic risks , including discrimination . These are important levers too , but it's a complex area , often involving difficult tradeoffs between , say , fairness for different groups and overall accuracy .
Speaker 1The report also discusses incorporating a behavioral approach into AI policy . What does that actually mean ?
Speaker 2It means leveraging insights from behavioral science . You know how humans actually make decisions , including all our cognitive biases and shortcuts , to inform the design and regulation of AI .
Speaker 1How so .
Speaker 2Well understanding human cognitive biases can help policymakers design rules that protect users from being exploited by AI systems , particularly advanced agentic AI that might learn and exploit individual user preferences or vulnerabilities in detail .
Speaker 1So using insights into human behavior to make AI policy more effective , more protective .
Speaker 2Exactly Leveraging those insights for good , while also limiting the ways AI itself can use those insights for potentially manipulative purposes . Interestingly , the report also notes the flip side . In certain domains , like maybe medicine or law , AI decision making could potentially overcome some of the cognitive biases humans are prone to . Oh right . Research is really needed there to understand when and how AI might actually make less biased decisions than humans and when human oversight or control remains absolutely essential .
Speaker 1Privacy and data protection , especially under the GDPR in Europe . This must be a huge challenge , given Gen AI's need for massive data sets .
Speaker 2It absolutely is . The report talks about Gen AI's insatiable appetite for data for training . This raises fundamental issues about data quality . If the training data is inaccurate or biased , it leads directly to harmful AI outputs . But there's also the risk of inference , where Gen-AI systems can infer sensitive personal information about individuals from seemingly innocuous , non-sensitive data they process process . The report mentions a retail example where analyzing shopping patterns allowed a company to infer quite accurately that a customer was pregnant , even though she hadn't told them . Wow .
Speaker 1How does GDPR apply here , then , and what are the practical challenges ?
Speaker 2It raises really complex questions about the lawfulness of processing such vast quantities of data , particularly whether legitimate interest is a sufficient legal basis for training models on these broad , often scraped data sets . Accountability is also challenging . Who is responsible when an AI system produces harmful or privacy-violating outputs ? The user , the provider . And implementing core data subject rights under GDPR , like your right to access your data or the right to have it erased , is technically incredibly difficult , maybe impossible sometimes , for these large , complex , opaque AI models . How do you find and delete one person's data from a model trained on trillions of data points ?
Speaker 1Are the regulatory bodies , like data protection authorities , getting involved ?
Speaker 2Yes , absolutely . Dpas in countries like Italy and the Netherlands have already taken action regarding specific Gen AI services . Italy and the Netherlands have already taken action regarding specific Gen AI services . The European Data Protection Board , the collective body of EU DPAs , clearly sees its supervisory role extending to Gen AI processing personal data , and it's calling for close cooperation with the new EU AI office set up under the AI Act .
Speaker 1But we don't have all the answers yet .
Speaker 2No , no , as the report notes , definitive legal and technical answers for many of these issues , especially around model opacity and actually implementing data subject rights effectively . They're still very much needed . It's an ongoing process .
Speaker 1OK , finally , let's tackle the copyright challenges .
Speaker 2This is , you balance protecting the rights of creators whose work exists today with enabling the innovation of AI , which often seems to require access to vast amounts of existing content for training .
Speaker 1So what's the key legal issue ?
Speaker 2A key one in the EU is the application of the text and data mining TDM exception . This comes from the EU's copyright directive . It basically allows TDM on lawfully accessed works unless the right holder has explicitly reserved their rights in an appropriate manner , especially using machine-readable means .
Speaker 1And what counts is appropriate machine-readable means . Is that clear ?
Speaker 2Well , that's a major part of the debate . In the ongoing litigation there's uncertainty whether just putting a line in your terms of service is enough or if more technical measures are required , things beyond just the standard robotstxt
Privacy, Copyright, and Regulation
Speaker 2file , which has limitations . Efforts like AIPF are trying to develop better standards , but it's not settled . Court cases in Germany , the Netherlands , the US , they're all grappling with this interpretation right now .
Speaker 1What about the content the AI actually produces ? Can that be copyrighted itself ?
Speaker 2Generally under EU copyright law , for something to be protected , it needs to show originality and reflect the author's own intellectual creation . That typically requires human input , human creative choices .
Speaker 2So if it's just generated automatically , If an AI output is generated purely automatically , with minimal human direction , it's unlikely to qualify for copyright protection itself . If a human user provides sufficiently precise instructions or makes significant creative selections or modifications that shape the final output , then that output might be protected , but the human user would need to demonstrate their specific creative contribution . Courts in different countries are starting to rule on what level of human intellectual input is actually sufficient .
Speaker 1And if the AI's output infringes someone else's existing copyright , who's liable then ? The user , the AI company ?
Speaker 2Infringement can definitely occur , For instance , if the AI model has effectively memorized parts of its training data , maybe a specific image or text passage , and reproduces it too closely in its output . If the AI provider didn't have the rights to use that training material for that purpose in the first place , there's a problem .
Speaker 1And who gets sued ?
Speaker 2Liability could potentially fall on the user who prompted the infringing output or on the AI provider or both . But most of the recent high profile cases we've seen in the US and Europe like the New York Times lawsuit against OpenAI , cases brought by collecting societies like GEMA in Germany or publishers in France against major AI companies they have primarily targeted the AI providers themselves .
Speaker 1Are there solutions being explored ? To try and navigate , says maybe find a middle ground .
Speaker 2Yes , things are starting to happen . Licensing agreements are emerging between some AI providers and publishers , like media organizations , allowing the AI to train on their content , usually for a fee . Collective licensing and collective bargaining are also being discussed as potential mechanisms , though they're complex to set up , to ensure fair compensation for creators whose works are used in training .
Speaker 1So finding ways to pay for the data .
Speaker 2Essentially , yes , finding workable models . Ultimately , the report suggests that harmonized approaches and standardization across the EU are really necessary here to provide legal certainty for everyone creators , users and the AI developers .
Speaker 1Wow , that was an incredible deep dive . We covered so much ground , from the basic tech all the way through these really complex policy challenges across technology , the economy , society , ethics , regulation , everything .
Speaker 2The EU is clearly grappling with how to foster innovation in this space , how to become a leader while also upholding its core values and ensuring a strategic , evidence-based approach , and that's where reports like this one from the JRC are so vital .
Speaker 1Absolutely
Conclusion and Future Considerations
Speaker 1. The insights from this kind of scientific evidence are clearly crucial for navigating this incredibly fast changing landscape .
Speaker 2And that evidence is vital for policymakers trying to make informed decisions in a world where , frankly , the technology is often evolving much faster than regulations can keep up .
Speaker 1So , as we wrap up this deep dive into the JRC's outlook on generative AI , here's maybe a thought to leave you with , building on everything we've discussed . Here's maybe a thought to leave you with , building on everything we've discussed . Given Gen AI's incredible capacity to generate content , to automate , to augment human tasks , how do we ensure its widespread adoption doesn't inadvertently diminish essential human skills , skills like critical thinking , creativity , nuanced understanding , especially in sensitive areas like education , journalism , public discourse ? How can we design AI systems and the policies governing them so they truly augment human capabilities and human values , rather than eroding them ?
Speaker 2That's a fundamental question , isn't it ? Especially as this technology continues to integrate deeper and deeper into our lives .
Speaker 1Indeed Well . Thank you for joining us for this deep dive . We hope you feel a little more well-informed about the fascinating and definitely challenging world of generative AI .
People on this episode
Aayushi Ramakrishnan
Co-host
Arjun Radeesh
Co-host
Imnah Varghese
Co-host
Lovell Menezes
Co-host
Radhika Mathur
Co-host
Sukayna Kazmi
Co-host
Stephen King
EditorPodcasts we love
Check out these other fine podcasts recommended by us, not an algorithm.
The Agentic Insider
Iridius.ai Inc
Brew Bytes - The Practitioner's AI Show
Nadeem Ibrahim
