One of the ways we find out our online activity is under surveillance is when an ad for a particular activity or product we search for magically appears on our social media feed.

It probably means a third-party tracking pixel has been following us around, noting the webpages we visit and recording what we click on.

Tracking cookies can do this as well, but they can be deleted and blocked, and they’re fading out of use.

No so with tracking pixels. They’re buried deep in a website’s coding and are often shared with social media companies so they can target you with ads based on your online activity. When it’s an item of clothing or a new pair of headphones you’ve been googling , that’s one thing, but it can get a lot more personal.

Long-held privacy concerns

The Office of the Australian Information Commissioner (OAIC), which houses the Privacy Commissioner, has long held concerns about how tracking pixels can pry into our personal business, especially when it comes to matters of health.

In late 2024, the OAIC took a close look at 50 health services websites, including providers of mental health services, healthcare for children, pharmaceutical products and fertility services.

Nearly all (96%) used tracking technologies, and more than half (52%) used third-party tracking pixels. In this latter group, more than three-quarters (77%) neglected to disclose the fact that they were using tracking pixels in their privacy policies.

The OAIC, pursuing what it calls a “risk-based and proportionate approach” to regulation, then narrowed the list of 50 websites down to 12 based on the sensitivity of the information the tracking technologies were collecting and the health services being offered.

All 12 websites used more than one third-party tracking pixel, including one provided by Meta in every case. Half the sites used the TikTok tracking pixel, while a quarter used one provided by Snapchat.

Tracking your every click

Several aspects of a visitor’s browsing activity were transmitted to social media platforms via these tracking pixels, including the full URLs of the webpages visited, button clicks, website searches, and time and date stamps – the kind of data used to build profiles and direct targeted ads.

Very few people would have had the knowhow to detect that their web browsing activity was being monitored, or that their browsing activity was being shared with their social media providers.

The OAIC investigation revealed that the information transmitted via tracking pixels “can reveal incredibly intimate details about an individual’s life”, including any health conditions you may have, the support services you’re searching for (such as mental health or domestic violence support), which services are recommended, and which medications you add to your cart when shopping on a health provider website.

Companies unaware pixels were in use

When the OAIC contacted the 12 health providers, several convincingly claimed that they were not aware tracking pixels were active on their websites.

In one case, an external website design company had left 50 tracking pixels on a health provider’s website without its knowledge, which were removed once the OAIC got in touch. The company had not been aware of the type of information that was being shared with external platforms.

Other health providers knew about the tracking pixels on their websites but assumed any information they harvested from visitors was “hashed”, or turned into digital characters that de-personalise information. This was not the case.

CoCare, a community-based health service provider that specialises in mental health and chronic illness support, added Meta and TikTok pixels to its website on the advice of a marketing firm. Through these platforms, it targeted its first ad campaign at people between the ages of 25 and 30 who had visited its website in the last two weeks. It gleaned their approximate ages and other personal information via the tracking pixels.

Tracking pixels found to breach privacy

In late June this year, the OAIC determined that the use of tracking pixels by two health providers, Medmate (telehealth) and Monash IVF (fertility services), amounted to a violation of the Privacy Act because the website visitors did not consent to having their information used for advertising. The investigation took a year to complete.

“Australians have become accustomed to pervasive online tracking and targeted advertising, but that doesn’t mean that they’re comfortable with it,” says Privacy Commissioner Carly Kind.

“In particular when it comes to targeted advertising based on sensitive data, our community attitudes research shows that nine in 10 Australians consider it neither fair nor reasonable to be targeted on the basis of their sensitive health data.”

The OAIC’s decision “establishes that the advanced technology used for tracking and targeting in the online realm still has to be used in compliance with the Privacy Act. That means website providers must obtain consent where they’re using tracking pixels to collect sensitive information, such as data on health, political opinions, race or ethnicity”.

How to detect tracking pixels

• Free online tools are available that will scan websites and detect the tracking technologies in use. The OAIC has highlighted , which was developed by investigative journalists. 
• The European Data Protection Board also offers , though it requires a level of tech proficiency that some may not possess. 
• Browser extensions can inspect websites for tracking technologies, including well-established tools such as , , and .
• Many social media platforms allow users to view and download the data they have on them. Meta’s , for instance, allows you to see the information that other businesses have shared about you. TikTok users can access such information by selecting Settings and Privacy > Account > Download your data. You can also clear past activity on many social media accounts and adjust settings to prevent your browsing activity across the internet from being used for targeted advertising on the social media platform. 

Get the inside story on our investigations into consumer rip-offs and bad business practices.

Sign up

Read our privacy policy

The post Healthcare services violated privacy by enabling targeted ads appeared first on ����Vlog.

There was once a time when the internet was not dominated by a handful of increasingly powerful companies making untold billions.

But that was before we realised how much data we’ve been giving away and what the big companies were doing with it.

If you’ve wondered how much this data is really worth, a research firm has finally put a number on it. It’s a lot of money.

The Switzerland-based Web3 Foundation reports that the commercial value of an average internet user’s data over their lifetime is USD$160,000 (roughly $225,000 in Australian dollars). 

That value adds up to a much bigger amount – $745 trillion – if applied to the world’s six billion internet users over a period of 60 years, which was the researchers’ chosen timeframe. 

Big names like Amazon, Alphabet (Google), Microsoft, Meta and Anthropic are especially adept at monetising our personal information.

This is not just ad revenue we’re talking about: it’s the commercial value of our online searches, clicks, locations, purchases, prompts, messages, images, preferences and behavioural signals. It’s all grist for the mill.

“For too long, the internet has operated on an implicit bargain that users do not fully understand – convenience in exchange for surveillance,” says Web3 founder Gavin Wood.

“The modern digital economy is powered by human data, yet the people generating that value have little visibility, control or participation in the upside.”

Toward a user-led internet

Web3 Foundation – which says it develops decentralised web software “with a focus on giving users greater control over their identity, data, digital assets and online interactions” – admits that the numbers are more of a benchmark than a precise calculation.

But the company’s report, The Hidden Price of Free: What Your Data Is Really Worth, comes with a detailed explanation of its methodology, which involved assessing the practices of 129 major tech companies.

“For decades, digital platforms have been built around centralised control, where users hand over their data, identity and value in exchange for access to services,” says Web3 vice president of technical operations Bill Laboon. “The internet does not have to work this way.”

He adds that as “AI accelerates and data becomes even more valuable, building a more transparent, user-led internet is becoming increasingly urgent”.

The report breaks down the revenue calculations, going from lowest to highest, into ‘conservative’, ‘central’ and ‘expansive’ scenarios.

The conservative scenarios of Personal Data Annual Value are eye-opening enough: 

• Globally – $485 per person per year
• USA – $4816 per person per year
• North America – $3390 per person per year
• UK and Europe – $938 per person per year
• Rest of the world – $155 per person per year

The Web3 report declares itself “a vital benchmark for understanding the scale of commercial value associated with personal data”.

“It shows that the current internet economy depends on a vast transfer of value from individuals to companies, usually without meaningful visibility, bargaining power, compensation or control”, the researchers write.

Only 4% of Australians trust AI

The global extrapolations of the value of our data come at a time when trust of its most powerful processor, AI, has hit an all-time low. 

According to the latest Australian Community Attitudes to Privacy Survey (ACAPS) report released by the Office of the Australian Information Commissioner (OAIC) in late May, only 4% of Australians trust AI companies to protect their privacy, and only 3% trust social media platforms. 

Australian Privacy Commissioner Carly Kind says the findings show that the AI sector has a long way to go to get the right privacy protections in place.

The ACAPS report “points to a community that places a high value on privacy, but does not consistently experience privacy protections as workable in practice,” Kind says.

“The survey’s findings mirror the rise in privacy complaints received by the OAIC – which have increased by 73% this financial year to date. Our efforts are delivering positive outcomes for the community, including speedier complaint timeframes, but community concern continues to grow at an alarming rate.”

• Only 1 in 10 respondents say organisations’ online practices are usually fair, while 35% say they are mostly or always unfair.
• Around two-thirds of Australians (68%) say they would be more likely to use digital services requiring personal information if they knew their data was handled fairly and responsibly.

The post How much money is big tech making from us? appeared first on ����Vlog.

Not so long ago, your online search history and other personal data were needed for the digital masters to assemble a profile of you for advertising purposes.

Now a team of researchers from the University of New South Wales (UNSW) and Queensland University of Technology (QUT) have discovered that artificial intelligence (AI) doesn’t need such old-fashioned data points. It can suss you out just from the online ads you’re shown.

“The key point is that the ads a person sees are not random,” says lead author of the study, Baiyu Chen of UNSW. Its title serves as yet another warning about how AI can be misapplied: When Ads Become Profiles: Uncovering the Invisible Risk of Web Advertising at Scale with LLMs.

(LLMs are Large Language Models, an AI term for computer systems that can analyse and recreate massive amounts of human language and approximate thought.)

“Advertising systems optimise delivery based on inferred profiles and behaviours, so the overall pattern of ads shown to a user can carry signals about traits such as gender, age, education, employment status, political preference, and broader socioeconomic position,” Chen says. “Our study shows that LLMs can analyse those patterns and infer private attributes from ad exposure alone.”

The research team – which also included professor Flora Salim, professor Daniel Angus, Dr Benjamin Tag and Dr Hao Xue – are part of the ARC Centre of Excellence for Automated Decision-Making and Society (ARC ADM+S).

The study is the result of the organisation’s Australian Ad Observatory project, which involved analysing around 435,000 Facebook ads seen by 891 Australian users. The findings mean AI can sift through markers of our selves we didn’t realise were there, and do it 200-times cheaper and 50-times faster than humans.

But what’s really concerning is that AI systems can build convincing profiles of us based on this targeted ad data after we browse the internet for just a short while, and they can infer personal characteristics as well as – or better than – humans.

Popular LLMs available to us all “can accurately reconstruct complex user private attributes”, the researchers write. And while the AI versions of who we are, based on the ads we see, may not capture everything, “they are often close enough to reveal meaningful insights about a person’s life stage or financial situation”.

Traits encoded in algorithms

All of this increases our risk of being scammed, since AI tools analysing ads can bypass current privacy protections, create profiles of us, and allow scammers to personalise their attacks.

Major digital platforms such as Google, Meta, TikTok, and X restrict certain sensitive categories of information from being used in targeted advertising, including an ad target’s health status, gender, sexual orientation, financial situation and political affiliation. 

But these personal traits are still encoded in the algorithms that drive the ads on these platforms, and widely available AI models can extract this information.

“Everyday tools such as browser extensions could be repurposed to quietly collect ads and build detailed user profiles — bypassing platform safeguards and leaving little trace,” the researchers report.

The vulnerable extensions – including ad blockers, coupon finders, and page translators – provide “perfect cover for data harvesting”, the researchers write. It means a cybercriminal skilled at AI tools wouldn’t have to hack your device or obtain personal information from you in order to set up a targeted scam.

Policy must evolve

The researchers argue that policy and regulation “must evolve to address not just data collection, but what can be inferred from the content people are exposed to”.

Chen says platform users can reduce the risks by limiting the use of browser extensions, limiting unnecessary permissions, and using any available privacy and ad-personalisation settings in their web browser settings.

But the platforms are going to have to tackle the new privacy risks brought on by the age of AI as well.

“This is not something users can fully solve on their own, because the broader issue is systemic,” Chen says. “People cannot easily opt out of the ad ecosystem altogether, so stronger platform safeguards are also needed.”

In other words, we’re leaving digital footprints for AI to follow every time we go online, and what will be done with this record of our personal attributes is anybody’s guess.

Get the inside story on our investigations into consumer rip-offs and bad business practices.

Sign up

Read our privacy policy

The post The ads you see reveal a lot about you – and AI is watching appeared first on ����Vlog.

When we surveyed 1000 renters, 41% said they felt pressured by their rental agent or landlord to use a RentTech platform such as Ignite, 2Apply, Snug, tApp or others.

Six out of 10 were uncomfortable with the amount and type of information being collected, and 29% decided not to use the platform to apply for this very reason.

RentTech constitutes a classic power imbalance, with finding a place to live at stake. The ultimatum imposed on renters is stark: give us the information we ask for or you can’t apply for a home.

Now the Australian Privacy Commissioner has taken action against 2Apply after a year-long review of its practices, which included asking prospective renters’ to reveal their gender, student status, citizenship status and visa expiry, as well as details of their previous rental history – information that goes well beyond what would be needed to assess the suitability of a tenant.

“Renters often lack real choice when making rental applications. Either they hand over personal and private information, including ID documents and payslips, or risk housing precarity or even loss,” says Privacy Commissioner Carly Kind.

“This not only places them at risk that their applications will not be considered fairly and equitably, but that their personal information may be compromised in a data breach or cyber-attack.”

Manipulative tactics called out

The Privacy Commissioner took a novel approach in determining that 2Apply was violating applicants’ privacy, considering the “design, structure and way information is conveyed in the 2Apply form” as well as utilising the concept of “online choice architecture”, which is about how “the presentation and structure of choices presented to individuals can shape how they make decisions”.

2Apply was found to have been applying a variety of manipulative techniques in its rental applications, including:

• “Confirmshaming” – the use of emotive language to make a user feel guilty or embarrassed for not taking an action that is beneficial to the information collector. 
• “Biased framing” – presenting choices in a way that emphasises their supposed benefits or disadvantages.
• “Bundled consent” – requesting consent for personal information to be used for multiple purposes in a single request. 

Can renters be forced to use a RentTech platform?

As ����Vlog recently reported, tenancy laws in some states require that renters are able to access an alternative payment and contact method.

In New South Wales and Victoria, tenants must be provided options like EFT or Centrepay. In Queensland, renters must be given a “reasonably accessible” option to pay rent.

But in practice, renters are in no position to push back against agents or landlords who breach these rules, as many do.

The essence of the Privacy Commissioner ruling is that 2Apply, which is operated by InspectRealEstate, violated the Australian Privacy Principles by collecting excessive personal information “by unfair means”, meaning prospective tenants felt they had no choice but to provide it.

The Office of the Australian Information Commissioner – which includes the Privacy Commissioner – has the power to issue fines up to $66,000, but 2Apply only had to agree to discontinue its excessive information collection, without admitting any fault.

In her determination on the case, Ms Kind emphasised the need for other RentTech platforms to note the ruling and change their information-gathering practices accordingly.

The post Rent platform 2Apply ordered to stop demanding excessive personal info from renters appeared first on ����Vlog.

An artificial intelligence (AI) tool developed by Elon Musk’s company xAI was recently banned in Indonesia and Malaysia and has raised serious concerns globally. It’s called Grok, and it gives users the capability to make highly sexualised images of people that look disturbingly real. 

As Indonesia’s Communication and Digital Affairs Minister Meutya Hafid recently put it, “The government sees non-consensual sexual deepfakes as a serious violation of human rights, dignity and the safety of citizens in the digital space.”

Britain’s media regulator, Ofcom, realeased a statement that says, “There have been deeply concerning reports of the Grok AI chatbot account on X being used to create and share undressed images of people – which may amount to intimate image abuse or pornography – and sexualised images of children that may amount to child sexual abuse material.”

Grok, which is free to X users who pay for a subscription, was launched in 2023, but in 2024 an image generator feature was added that included something called ‘spicy mode’, which can generate pornographic content.

Australia’s eSafety Commissioner says the agency “has seen a recent increase from almost none to several reports over the past couple of weeks relating to the use of Grok to generate sexualised or exploitative imagery”, adding that it “will use its powers, including removal notices, where appropriate and where material meets the relevant thresholds defined in the Online Safety Act”.

Malicious content made easier

Abhinav Dhall, an associate professor at Monash University’s Department of Data Science and AI, says Grok has put powerful new technology into the hands of wrongdoers.

“Grok has made it easier to produce malicious content because it is directly integrated into X, so anyone can quickly tag it and request image edits. As it is so well integrated into the platform, the edited outputs also appear directly within the same public thread, which increases the visibility and reach of manipulated images”, Dhall says, adding that in many cases “the original poster may not even have the rights to the image they are uploading on the platform, which can make it easier for the edits to become potentially defamatory or unsafe”.

Dhall says Grok users should take steps to avoid images falling into the wrong hands.

“To reduce the risk of personal images being used to generate malicious content, users should be careful about posting clear, front-facing photos of their face, and should check and tighten privacy settings on their social media platforms,” Dhall says.

“It is also important to avoid posting children’s photos publicly. If you suspect your images have been misused, reverse image search can be applied to detect AI-generated content, and fake or harmful content should be reported to the relevant platforms as quickly as possible.”

X said in a previous statement that it removes illegal content from its platform including child abuse material and suspends the accounts of people who post it.

Musk has posted comments on the Grok backlash, saying critics of X “just want to suppress free speech”. In an X post on 15 January he said, “Grok is supposed [to] allow upper body nudity of imaginary adult humans (not real ones) consistent with what can be seen in R-rated movies on Apple TV.”

In a more recent announcement on X the company said “we have implemented technological measures to prevent the Grok account from allowing the editing of images of real people in revealing clothing” in jurisdictions where this is illegal.

But it remains unclear how the company will block certain locations from using this functionality or which locations they may be.

Will mandatory codes stop the deepfakes?

On 9 March 2026, mandatory codes come into effect in Australia which impose new obligations on AI services to limit children’s access to sexually explicit content as well as to violent material and content related to self-harm and suicide. But enforcing such codes on mammoth AI companies based in the US and other countries has proven to be a tall order for Australian regulators.

Abhinav Dhall stops short of recommending that Grok be banned in Australia, saying it’s a matter of enforcing the current rules and compelling tech companies to stop harmful content.

“Australia already has laws covering image-based abuse, so the focus should be on making the penalties clear and ensuring it is easy for victims to report abuse and have content removed quickly,” Dhall says. “At the same time, social media platforms should be required to implement stronger guardrails to stop harmful edits before they spread.”

Meanwhile, amid the outcry around the world about sexualised deepfakes, in a speech given at Musk’s company SpaceX, in South Texas, US Defense Secretary Pete Hegseth recently said that the Pentagon will embrace Grok along with Google’s generative AI engine. 

The post What is Grok and should Australia be blocking it? appeared first on ����Vlog.

Very few of us read the privacy policies we passively consent to when engaging with a service provider. Fewer still would understand what these privacy policies actually say.

In recent years, ����Vlog has conducted several investigations that focused on the far-reaching permissions these documents give the businesses we regularly interact with.

In 2023, we reported on the privacy policies of rental platforms such as realestate.com.au’s Ignite as well as Ailo, Tenant Options, Rental Rewards, Snug, 2Apply and Simple Rent.

The conclusion? These RentTech platforms collected information that went well beyond what’s needed to assess a tenant’s ability to pay the rent. The questions often seemed designed to grab as much data as possible from people who had no choice but to provide it.

In 2024, we analysed the privacy policies of Australia’s ten most popular car brands to see how the vehicles monitored and tracked their drivers. Here again we found that the harvesting of personal driver information was often excessive, and the rights the manufacturers gave themselves to share the data with third-parties were both far-reaching and vague.

The ACCC has estimated that it would take the average Australian 46 hours to read all the privacy policies they encountered in a month, the average length of which is about 6876 words.

All of this makes the Office of the Australian Information Commissioner’s (OAIC) recent announcement that it will begin its first large-scale review of privacy policies in early January 2026 more timely than ever.

What’s changing in privacy law?

The Privacy Act requires privacy policies to contain certain details, such as what information is collected, why it’s needed, how it’s used, and how it can be corrected if necessary. 

An update to the Act in 2024 means businesses will also be required (as of 10 December 2026) to specify in their privacy policies whether a computer program will be using your personal information to make decisions that could go against you, such as when an application for a rental home is rejected. 

In addition, the 2024 update gave the OAIC the power to issue infringement notices for Privacy Act violations without going to court. And it gives individuals the right to seek legal redress and financial compensation in certain cases for invasions of privacy or misuse of their personal information.

The OAIC’s privacy policy sweep is taking a different approach than our investigations of online privacy documents. It will occur in the real world, focusing on information demanded by businesses in person, such as when a real estate agent asks you for personal details when you’re inspecting a rental property or a car rental company presents you with a lengthy form before handing you the keys. The privacy policies of such businesses must include the above-mentioned information. 

Not having the right information in a privacy policy – or not having a privacy policy at all – could lead to fines from the OAIC of up to $66,000.

Which types of businesses will be targeted?

The privacy policy sweep will focus on sectors where the OAIC believes there are particular power imbalances – also known as information asymmetries – between the business in question and the customers being asked to provide the information.

“When confronted with in-person requests for their personal information from retailers, licensed venues, car hire companies or real estate agents, consumers often don’t have access to all the information they might need to make an informed decision,” says Privacy Commissioner Carly Kind.

“This makes them vulnerable to overcollection of personal information and creates risks to their security and privacy.”

The OAIC says it will review the privacy policies of around 60 businesses from the following six sectors, with a particular focus in each case.

• Rental and property – collection of individuals’ personal information during property inspections.
• Chemists and pharmacists – collection of personal information for the purpose of providing a paperless receipt and collection of identity information to provide medication.
• Licenced venues – collection of identity information to enable individuals to access a venue.
• Car rental companies – collection of identity and other personal information to enable an individual to enter into a car rental agreement.
• Car dealerships – collection of personal information to enable an individual to conduct a vehicle test drive.
• Pawnbrokers and second-hand dealers – collection of identity information from individuals who wish to sell or pawn goods.

Transparent communication is critical

In the OAIC’s view, a business’s explanation of how it will use personal information should be open and transparent.

“The Australian community is increasingly concerned about the lack of choice and control they have with respect to their personal information,” Kind says.

“The first building block of better privacy practices is a clear privacy policy that transparently communicates how an individual can expect their information to be collected, used, disclosed and destroyed.

“In conducting a compliance sweep, the OAIC intends to ensure that entities are meeting their obligations to be transparent with consumers and customers about how they’re using the personal information they collect in-person.

“We hope this will also catalyse some reflection about how robust entities’ privacy practices are, and whether more can be done to improve compliance with the Privacy Act writ large.”

The post Real estate agents, chemists, car hire companies and more under new privacy scrutiny appeared first on ����Vlog.

It seems the artificial intelligence (AI) industry is grabbing our personal information as fast as it can – before regulations end up enforcing what the community actually wants. 

Among other indicators, the final report of the Australian Competition and Consumer Commission’s (ACCC) Digital Platform Services Inquiry, released in March this year, confirmed that the vast majority of us don’t want our personal data used to train AI tools without our consent. 

In a consumer survey that was part of the report, 83% of Australians said our approval should be mandatory. This aligns with research from the agency tasked with protecting our personal information, the Office of the Australian Information Commissioner (OAIC), which found that 84% of Australians want to have control over their personal information, including the right to demand that it be deleted. 

In mid-October, Roy Morgan published research showing that 65% of Australians think AI “creates more problems than it solves”, though 12% hailed its ability to advance medical science. 

There is considerable tension between those seeking to protect our data and those that see it as the key to improving products and services – and boosting profits – especially the big tech companies.  

Agencies of the federal government are also on board with this notion. According to a recent report by the Productivity Commission (PC), our personal data shouldn’t be locked away behind a wall of inflexible regulations. 

The Privacy Act’s requirements “are constraining innovation without providing meaningful protection to individuals”, the report says. It concludes that making it easier to harness data could add up to $10 billion a year to the country’s economy.

To that end, the PC calls for exempting some businesses from the requirement to obtain informed consent before accessing a person’s data, one of the key pillars of the Privacy Act. In the PC’s view, some businesses shouldn’t have to do this as long as they commit to acting in the person’s best interests when it comes to handling their privacy.  

The report goes on to argue that giving consent has become a meaningless exercise, since no one has the time to read whatever they’re consenting to. 

While businesses are required to have a privacy policy under the Privacy Act, these sprawling documents are all but impossible to understand. The ACCC has estimated that it would take the average Australian 46 hours to read all the privacy policies they encountered in a month, the average length of which is about 6876 words. 

These consent protocols are clearly not working as intended, but Privacy Commissioner Carly Kind has taken issue with the idea of watering down privacy regulations, making the case that protecting privacy and boosting productivity are not mutually exclusive and that informed consent is a critical consumer right. 

Patient scans used to train AI without consent 

But there have been cases in which the OAIC – where Kind is one of three commissioners – has allowed businesses to surreptitiously harvest our data to their own ends. 

In September last year, for instance, on the case of Australia’s biggest radiology chain, I-Med Radiology Network, entering a joint venture with the start-up AI platform Harrison.ai in 2019.

The plan was to use I-Med’s patient scans to train an AI tool called Annalise.ai, which has now reportedly been approved for use in over 40 countries and heralded as a gamechanger.

The quantity and quality of the I-Med data – around 30 million patient scans and the resulting diagnostic reports from Australia and other countries – was key to the tool’s success.

It is clearly a money-maker for the business. In April last year, the Australian Financial Review reported that the business was on track to record $1.35 billion in revenue for the financial year. 

(Harrison.ai’s flagship product was an AI model that was trained on 800,000 chest x-rays sourced from one of I-Med’s 270 clinics in Australia, Crikey reported. It owns Annalise.ai, which is now called Harrison.ai Radiology.) 

The Crikey article says there is no evidence that informed consent was obtained from the patients. Based on the above-mentioned surveys, this would likely have run counter to many of their wishes. Neither I-Med nor Harrison.ai have disputed this. 

But it appears there is some leeway on the informed consent requirement. In July this year the OAIC ruled that I-Med had not contravened the Privacy Act since the data had been de-identified and could no longer be defined as personal information.

The ruling suggests that, where privacy issues are not at play, it’s not the OAIC’s role to prevent businesses from grabbing our data to train AI, with or without our approval. 

An OAIC spokesperson tells ����Vlog that “the issue of consent is always a highly relevant factor”, but the protections of the Australian Privacy Principles (APPs) no longer apply when patient data is de-identified. 

“However, strong de-identification is challenging, and whether something is de-identified is context dependent. Data that is de-identified when subject to strict controls – as it was in the I-MED case – may not be de-identified in other contexts, such as if it is released publicly,” the spokesperson says. 

The right balance between privacy regulation and the development of new AI tools remains a work in progress.

While industry guidance from the OAIC stipulates that people should be informed when their data is being used to train AI, “consent is only required in limited circumstances by the APPs, and will not always be required when entities use personal information for AI training”, the spokesperson added. 

In other words, we have no copyright on the content of our bodies. Medical clinics are free to use our data for commercial purposes without telling us, and businesses can profit handsomely. 

The rise of AI scribes in GP consultations

All of this is pertinent to a related development – the growing use of AI tools by GPs and specialists to record patient visits, known as AI scribes. Currently around 22% of GPs are using AI scribes according to polling by the Royal Australian College of General Practitioners (RACGP). Popular models include Lyrebird, Heidi Health, Amplify+, i-scribe and Medilit, but there are many others. 

The RACGP has tentatively embraced some uses of AI scribes, especially where they can reduce the administrative burdens on doctors and free them up to concentrate more fully on preventative care. 

But the college has also expressed concern about AI tools being developed by profit-seeking tech firms without the oversight of medical clinicians. “Value to technology company shareholders might be prioritised over patient outcomes,” the RACGP wrote in a position statement on the issue. 

Of particular concern is the reliability – and the legality – of AI in making medical recommendations. The Therapeutic Goods Administration (TGA) has recently gone on record regarding the issue, saying medical professionals report that AI scribes “frequently propose diagnosis or treatment for patients beyond the stated diagnosis or treatment a clinician had identified during consultations”.

Such a functionality would mean AI scribes are medical devices that require pre-market approval, the TGA says. By dodging regulation, they are potentially being supplied to the medical industry in breach of the Therapeutic Goods Act. 

The RACGP hasn’t taken a position on whether AI scribes should be regulated by the TGA, but it does instruct GPs to obtain consent from patients before using them, to double check that the information they record is accurate, and to have a backup plan in place in case there’s a glitch with the AI scribe. 

“AI scribes can produce errors and inconsistencies and cannot replace the work GPs typically undertake to prepare clinical documentation,” a RACGP spokesperson tells ����Vlog. 

“GPs and other doctors must carefully check the output of an AI scribe to ensure its accuracy. Where an AI scribe performs its expected function – summarising information for independent GP review and decision making – it does not have a therapeutic use. Diagnosis, however, is outside the scope of an AI scribe.” 

Market use shouldn’t pre-date regulation

One very concerned citizen is AI expert Dr Kobi Leins, who was recently told to take her business elsewhere after asking that AI not be used during a specialist visit for her child. Liens cancelled the appointment, not least because she was familiar with the AI model in use and wasn’t impressed by its privacy and security features. She didn’t want it capturing her child’s data. 

“There is no need for many of these tools, and fundamentally, we need to ask why they are being pushed so hard and where money would be better spent in a healthcare system where doctors have time to listen to patients,” Leins says. 

“Individuals do not have the skills nor the capacity to review these tools. Regulatory bodies need to review them in a way that ensures privacy, manages risk and trains staff about their limitations and where they are safe to use. It’s about individual privacy, but also about group privacy. There are potentially grave harms based on racial and gender and other biases in the data these tools rely on.” 

Leins points to a recent study funded by the UK’s National Institute for Health and Care Research which found that when social workers used Google’s popular AI model ‘Gemma’, it downplayed women’s physical and mental health issues as compared to men’s in its case note summaries. In these cases, it’s the AI companies in the driver’s seat rather than healthcare workers, she argues. 

Leins is not alone. One concerned parent told us they didn’t trust the data handling practices of AI companies. 

How the data is managed “is dependent on your GP and the policy of whatever scribe they’re using, which you’re unlikely to know when they ask you to sign over consent,” the parent says. “And like with most companies, you have no control over what happens when they get hacked and all your personal health information ends up on the dark web. I generally consent for my data to be scribed, but not my kid’s.” 

The privacy policy of one of the most popular AI scribes, Heidi Health, is not reassuring. It says the business may share the personal information it captures with employees, third party suppliers, related companies, anyone it transfers the business to, ‘professional advisers, dealers and agents’, government agencies, law enforcement and more. Your data may also be transferred overseas. 

Heidi Health tells ����Vlog that the company doesn’t share patient-identifiable or health information with external parties except where required by law, that its data handling practices comply with the Privacy Act, and that it doesn’t use patient data to train AI.

“We understand that clinicians and patients will only embrace new technology when they have complete confidence that their data is secure and used responsibly,” says Heidi Health head of legal and regulatory affairs Yass Omar, adding that the company is the only AI scribe in Australia certified to ISO 27001, “a globally recognised standard that reflects the strength of our information security management systems”.

But concerns remain about AI scribes in general. Another parent recently encountered one while taking her daughter to a specialist. She was told the data would be deleted after it had been reviewed, “but I only had her word to go on. What are the privacy implications here? And what mistakes can and does it make? Does the doctor look at the summary after each appointment at the end of the day while it is fresh in their mind, to make sure that the scribe accurately captured the info?” 

Just like asking Dr Google 

Professor Enrico Coiera, who is the director of both the Centre for Health Informatics, Australian Institute for Health Innovation at Macquarie University and the Australian Alliance for AI in Healthcare, tells ����Vlog that one of his biggest concerns is that product development is far outpacing regulatory oversight. 

“Generative AI products in particular are being updated constantly. This makes it very hard for the traditional safety guardrails we rely on, like regulation, to make sure these new technologies are safe.

Much of this kind of AI is never marketed as ‘medical grade’, but as a general-purpose tool. So it is never assessed for its safe and effective use in healthcare.” 

Coiera says this is much like asking your search engine a health question. If an AI scribe is suggesting diagnosis or treatment, it’s a medical device that needs to be regulated by the TGA, he says. 

Referring to the I-Med case, Coiera says, “patients should be asked to consent to the use of their data for building AIs, and especially so if there is a risk their information is identifiable”. 

He recommends that patients read any consent forms carefully before signing. 

“If they are uncomfortable with their data being used for AI development, they should discuss that with their care provider.” 

Coiera is a believer in the capacity of AI to advance medical science, as long as people’s privacy is protected. 

“As long as I am comfortable that my data is stored securely and is de-identified before use, I would agree to its use for non-commercial research purposes.” 

The post AI is increasingly invading our medical privacy as regulation struggles to keep up appeared first on ����Vlog.

In 2022, ����Vlog reported that Kmart, along with Bunnings and The Good Guys were capturing the biometric data, or unique facial features known as a ‘face print’, of customers entering their stores. 

Our investigation prompted the Office of the Australian Information Commissioner (OAIC) to launch a probe into whether privacy laws had been breached with the Facial Recognition Technology (FRT).

In 2024, Privacy Commissioner Carly Kind found that Bunnings had breached the law and today an announcement was made that Kmart had done so too. 

Kmart sought to justify its use of FRT in stores between June 2020 and July 2022 as a measure to prevent refund fraud. However, the Commissioner said Kmart did not seek customer consent to collect biometric information and its collection was not proportional, as there were other means available to address refund fraud. 

“I do not consider that the respondent (Kmart) could have reasonably believed that the benefits of the FRT system in addressing refund fraud proportionately outweighed the impact on individuals’ privacy,” Kind says. 

No penalty 

OAIC did not seek a financial penalty against Kmart in this case, similar to the case with Bunnings last year. 

In a statement a Kmart spokesperson says they are “disappointed” with the ruling and are reviewing options to appeal the determination. 

“Like most other retailers, Kmart is experiencing escalating incidents of theft in stores which are often accompanied by anti-social behaviour or acts of violence against team members and customers,” the spokesperson says. 

Commissioner Kind says that despite the two rulings against Bunnings and now Kmart, FRT was not ‘banned’ in Australia. 

“The human rights to safety and privacy are not mutually exclusive; rather, both must be preserved, upheld and promoted. Customer and staff safety, and fraud prevention and detection, are legitimate reasons businesses might have regard to when considering the deployment of new technologies. However, these reasons are not, in and of themselves, a free pass to avoid compliance with the Privacy Act,” she stated.

The post Kmart’s facial recognition technology broke the law, commissioner rules appeared first on ����Vlog.

This article mentions suicide. If you or anyone you know needs support, contact Lifeline on 13 11 14 or at , or Beyond Blue on 1300 224 636 or at .

Early in 2024 Sarah* started using the generative artificial intelligence (AI) tool ChatGPT for small administrative tasks.

“I would just experiment with it, for the start of a business correspondence or instead of using Google for something I would use ChatGPT,” she says. 

After a while she began using it as a tool to supplement her mental health therapy, using the AI like a “sounding board” between her psychologist sessions. 

“I would tell the AI my attachment styles and ask it how best to respond to certain situations. I would ask it to keep an eye out for any red flags in the early stages of dating,” says Sarah. 

“You want to feel like you have been heard and articulating that to someone, or something else, can be really helpful, kind of like a diary.” 

Sarah felt like the AI chatbot helped supplement her therapy sessions rather than replace them, and says her therapist wasn’t discouraging about her use of it.

However, now Sarah is no longer seeing her therapist and is still using ChatGPT, while also experimenting with the new popular AI tool DeepSeek. 

“I do have concerns. There is [the issue of] privacy, and then also I think ‘am I living more of a bubble life in an echo chamber after COVID, is this just a further way of isolating myself?'” 

The rise of therapy chatbots 

While some people, like Sarah, are using general AI chatbots like ChatGPT, DeepSeek, MetaAI and Gemini for mental health-related assistance, there are also a rising number of AI products designed specifically to help with mental health.

The US for-profit sleep and meditation mental health app Headspace (not to be confused with the Australian youth not-for-profit organisation of the same name) is in the top ten mental health websites visited in Australia. In October last year, it launched Ebb, “an empathetic AI companion integrated into the app to help people navigate life’s ups and downs”. 

The Headspace app has reportedly been downloaded more than 80 million times across various app stores. 

Other major players in the AI therapy chatbot space include Wysa and Youper, both of which have over a million downloads in the GooglePlay Store. Youper says it has helped over 2.5 million users.  

Not a replacement for therapy

While mental health therapy apps don’t specifically claim to replace in-person psychologists, they do make varying claims about their ability to treat mental health conditions. 

Headspace’s Ebb

Headspace’s page promoting Ebb says it was developed by clinical psychologists using scientific-backed methods. 

“Ebb is an AI-powered tool that’s designed to help you better understand yourself. While therapy and coaching provide deeper emotional support, Ebb can help you maintain mental wellness by encouraging regular reflection and mindfulness,” the website says. 

Further down the website says, “Ebb is not a substitute for medical or mental health treatment. If you need support for a mental health condition, please talk with a licensed provider”.

Headspace tells ����Vlog that Ebb is not a “therapy chatbot” but rather “a sub-clinical support tool for our members to process thoughts and emotions or reflect on gratitude”. 

“We acknowledge that Ebb, like any AI tool, has limitations. It’s not intended to replace professional mental health care but to complement it by helping users manage stress, practice meditation, and engage in self-reflection,” a spokesperson says. 

The spokesperson added that the chatbot has ways of detecting high risk situations such as suicidal thoughts or self-harm ideation, and will suggest that users contact emergency services. 

Youper

The Youper website makes the point that the supply of mental health clinicians is not keeping up with demand for mental health services.

“The groundwork of Youper is evidence-based interventions – treatments that have been studied extensively and proven successful. Youper has been proven clinically effective at reducing symptoms of anxiety and depression by researchers at Stanford University,” the site claims,  citing  83% of users experience “better moods”.

Youper did not respond to our questions about the scientific underpinnings of these claims. 

Wysa

Wysa says its product is designed to complement traditional therapy and undergoes continuous compliance and safety testing to make sure users don’t receive harmful or unverified advice. 

“While ChatGPT and Gemini are powerful general AI models, Wysa is purpose-built for mental health with strict clinical safety guardrails,” the chief of clinical services and operations at Wysa, Smriti Joshi, tells ����Vlog. 

The unknown potential harms

Professor Jeannie Paterson, director for the Centre of AI and Digital Ethics at the University of Melbourne, says there is a lot we don’t know about how AI therapy chatbots function.

Paterson says there is a big difference between using the technology in collaboration with a professional and going to an app store to buy something we “know very little about”. 

“They could be causing you harm, or you’re paying for something that does you very little good, because it’s just not tested.” 

Previous media reporting around the world has highlighted some of the potentially harmful algorithms used in therapy chatbots when things go wrong.

In 2018, the BBC reported that therapy apps Wysa and Woebot, which were being promoted to children in the UK, failed to suggest to a journalist posing as a child who was being sexually abused that they contact emergency services and get help. Both apps have had significant updates since then. 

According to National Public Radio in the US, in an apparent cost-saving measure, the US-based National Eating Disorder Association fired their phone-line staff in 2023 and began promoting an AI chatbot instead, which went on to suggest dieting advice to people with eating disorders. 

Meeting a demand 

Piers Gooding, an associate professor at Latrobe University law school, has researched mental health chatbots extensively. 

He says he expects usage to continue to grow, simply because so much money is being poured into their development worldwide. Australia’s healthcare system is more effective than that in countries like the United States, but there are still sizeable gaps in demand. 

“In 2021, according to one market report, digital startups focusing on mental health secured more than five billion dollars in venture capital – more than double that for any other medical issue, and investment further increased in 2023,” he says. 

“I suspect there will be some kind of reckoning with some of the over-claiming about what they can do, and that might come in the form of people just voting with their feet and realising that it’s not quite what it’s cracked up to be.” 

Australian Association of Psychologists Inc policy coordinator Carly Dober says she understands that in a cost-of-living crisis, many people can’t afford to seek the mental health care and support they need and chatbots appear to be a more affordable option. 

“I don’t think it’s the fault of people for trying to find whatever they can to support themselves in the moment. But unfortunately, when there is that kind of vacuum, sometimes not the most helpful players will try to fill that market or that space,” she says.

Lack of regulation? 

Dober says there is a big difference between using a chatbot in conjunction with therapy, versus replacing therapy with AI, which she says lacks the “checks and balances” needed. 

“There is no uniform law around AI chatbots, there is a lack of regulation of that space, whereas we as psychologists are a highly regulated field,” she adds. 

TGA exemptions

Latrobe University’s Gooding says the Therapeutic Goods Administration (TGA) seems to provide an exemption to AI therapy chatbots being regulated as a “medical device” if the providers take steps such as:

• working from widely accepted cognitive behavioural therapy (CBT) models
• not providing experimental therapy 
• not diagnosing mental health conditions. 

The TGA says it is currently “reviewing the ongoing appropriateness” of these exemptions and that it will consult stakeholders before suggesting any changes to government. It adds that to date it has received no complaints about therapy chatbots.

‘Innovation important’

Gooding says that despite the lack of TGA regulation, under Consumer Law claims made by app providers about the benefits of their products must not be misleading or deceptive. 

He adds that innovation in the technology space is important and that a “heavy handed” regulatory approach from the TGA might not best serve consumers. 

“Regulatory change is almost certainly needed to remove the digital mental health app exemption in relation to chatbots. Chatbots can’t capture nuance, and they can be easily programmed to have addictive elements and pretend to be real people. That poses a real danger in the mental health context,” Gooding says.

*Not her real name

The post Should you trust an AI chatbot with your mental health? appeared first on ����Vlog.

In our increasingly data-driven world – where details on a database can define who we are– an administrative mixup by a government agency can easily morph into a recurring nightmare.

It’s an admin error that can happen if we share the same name and date of birth with someone – our digital doppelganger. Once our personal information gets mixed, there’s no telling how long it’s going to take to straighten it out to the government’s satisfaction. It’s also difficult to predict how long the information management failure will affect our lives.

Given that hundreds, if not thousands, of people in Australia share the same name and date of birth, it’s a serious issue. But it’s worth pointing out that government agencies are not supposed to let these mix ups happen, as a recent determination by the Office of the Australian Information Commissioner (OAIC) makes clear.

Focusing on an individual case, the OAIC ruled earlier this month that Services Australia had failed to prevent an individual’s Medicare information from being mixed up with – and disclosed to – another person, a violation of the Privacy Act.

Intertwined records can cause real harm

“The idea of a digital doppelganger might seem like a curiosity, but its catchy ring conceals a difficult reality,” says Privacy Commissioner Carly Kind in a recent blog on this issue. 

“Individuals whose government records, such as Medicare, Centrelink and child support services, are intertwined may suffer not only inconvenience but real harm. They or their health practitioners may be prevented from accessing accurate records to enable the timely provision of health services.”  

In the case mentioned above, the affected person’s records were intertwined with another person’s for six years, during which time he continually contacted Services Australia and tried in vain to clear things up.

At one point the agency assured the person that the matter had been resolved.

Then he got a message from Services Australia saying “your registered Safety Net family is getting close to reaching the Medicare Safety Net Threshold”, which was clearly meant for another person. 

Following that, he discovered that his COVID and influenza vaccination history had been assigned to another person. 

Kind acknowledges that Services Australia has taken steps since the incident to keep digital doppelgangers administratively separate from one another, but the possibility of a mixup still exists.

The OAIC awarded the complainant $10,000 in compensation “for distress arising from the privacy breaches, which must have been a considerable burden on his time and energy over the past decade,” Kind says.

The post Digital doppelgangers throwing lives into chaos  appeared first on ����Vlog.