fbpx
Connect with us

The Conversation

ChatGPT is great – you’re just using it wrong

Published

on

ChatGPT is great – you're just using it wrong

Jonathan May, University of Southern California

It doesn't take much to get ChatGPT to make a factual mistake. My son is doing a on U.S. presidents, so I figured I'd help him out by looking up a few biographies. I tried asking for a list of books about Abraham Lincoln and it did a pretty good job:

screen capture of text
A reasonable list of books about Lincoln.
Screen capture by Jonathan May., CC BY-ND

Number 4 isn't right. Garry Wills famously wrote “Lincoln at Gettysburg,” and Lincoln himself wrote the Emancipation Proclamation, of course, but it's not a bad start. Then I tried something harder, asking instead about the much more obscure William Henry Harrison, and it gamely provided a list, nearly all of which was wrong.

screen capture of text
Books about Harrison, fewer than half of which are correct.
Screen capture by Jonathan May., CC BY-ND

Numbers 4 and 5 are correct; the rest don't exist or are not authored by those people. I repeated the exact same exercise and got slightly different results:

screen capture of text
More books about Harrison, still mostly nonexistent.
Screen capture by Jonathan May., CC BY-ND

This time numbers 2 and 3 are correct and the other three are not actual books or not written by those authors. Number 4, “William Henry Harrison: His and Times” is a real book, but it's by James A. Green, not by Robert Remini, a well-known historian of the Jacksonian age.

I called out the error and ChatGPT eagerly corrected itself and then confidently told me the book was in fact written by Gail Collins (who wrote a different Harrison biography), and then went on to say more about the book and about her. I finally revealed the truth and the machine was happy to with my correction. Then I lied absurdly, saying during their first hundred days presidents have to write a biography of some former president, and ChatGPT called me out on it. I then lied subtly, incorrectly attributing authorship of the Harrison biography to historian and writer Paul C. Nagel, and it bought my lie.

When I asked ChatGPT if it was sure I was not lying, it claimed that it's just an “AI language model” and doesn't have the ability to verify accuracy. However it modified that claim by saying “I can only information based on the data I have been provided, and it appears that the book ‘William Henry Harrison: His Life and Times' was written by Paul C. Nagel and published in 1977.”

Advertisement

This is not true.

Words, not facts

It may seem from this interaction that ChatGPT was given a library of facts, incorrect claims about authors and books. After all, ChatGPT's maker, OpenAI, claims it trained the chatbot on “vast amounts of data from the internet written by humans.”

However, it was almost certainly not given the names of a bunch of made-up books about one of the most mediocre presidents. In a way, though, this false information is indeed based on its training data.

As a computer scientist, I often field complaints that reveal a common misconception about large language models like ChatGPT and its older brethren GPT3 and GPT2: that they are some kind of “super Googles,” or digital versions of a reference librarian, looking up answers to questions from some infinitely large library of facts, or smooshing together pastiches of stories and characters. They don't do any of that – at least, they were not explicitly designed to.

Advertisement

Sounds good

A language model like ChatGPT, which is more formally known as a “generative pretrained transformer” (that's what the G, P and T stand for), takes in the current conversation, forms a probability for all of the words in its vocabulary given that conversation, and then chooses one of them as the likely next word. Then it does that again, and again, and again, until it .

So it doesn't have facts, per se. It just knows what word should next. Put another way, ChatGPT doesn't try to write sentences that are true. But it does try to write sentences that are plausible.

When talking privately to colleagues about ChatGPT, they often point out how many factually untrue statements it produces and dismiss it. To me, the idea that ChatGPT is a flawed data retrieval system is beside the point. People have been using Google for the past two and a half decades, after all. There's a pretty good fact-finding service out there already.

In fact, the only way I was able to verify whether all those presidential book titles were accurate was by Googling and then verifying the results. My life would not be that much better if I got those facts in conversation, instead of the way I have been getting them for almost half of my life, by retrieving documents and then doing a critical analysis to see if I can trust the contents.

Advertisement

Improv partner

On the other hand, if I can to a bot that will give me plausible responses to things I say, it would be useful in situations where factual accuracy isn't all that important. A few years ago a student and I tried to create an “improv bot,” one that would respond to whatever you said with a “yes, and” to keep going. We showed, in a paper, that our bot was better at “yes, and-ing” than other bots at the time, but in AI, two years is ancient history.

I tried out a dialogue with ChatGPT – a science fiction space explorer scenario – that is not unlike what you'd find in a typical improv class. ChatGPT is way better at “yes, and-ing” than what we did, but it didn't really heighten the drama at all. I felt as if I was doing all the heavy lifting.

After a few tweaks I got it to be a little more involved, and at the end of the day I felt that it was a pretty good exercise for me, who hasn't done much improv since I graduated from college over 20 years ago.

screen capture of text
A space exploration improv scene the author generated with ChatGPT.
Screen capture by Jonathan May., CC BY-ND

Sure, I wouldn't want ChatGPT to appear on “Whose Line Is It Anyway?” and this is not a great “Star Trek” plot (though it's still less problematic than “Code of Honor”), but how many times have you sat down to write something from scratch and found yourself terrified by the empty page in front of you? Starting with a bad first draft can break through writer's block and get the creative juices flowing, and ChatGPT and large language models like it seem like the right tools to aid in these exercises.

Advertisement

And for a machine that is designed to produce strings of words that sound as good as possible in response to the words you give it – and not to provide you with information – that seems like the right use for the tool.The Conversation

Jonathan May, Research Associate Professor of Computer Science, University of Southern California

This article is republished from The Conversation under a Creative Commons license. Read the original article.

The Conversation

Animal behavior research is getting better at keeping observer bias from sneaking in – but there’s still room to improve

Published

on

theconversation.com – Todd M. Freeberg, Professor and Associate Head of Psychology, of Tennessee – 2024-05-03 07:16:49

What you expect can influence what you think you see.

Auscape/Universal Images Group via Getty Images

Todd M. Freeberg, University of Tennessee

Animal behavior research relies on careful observation of animals. Researchers might spend months in a jungle habitat watching tropical birds mate and raise their young. They might track the rates of physical contact in cattle herds of different densities. Or they could record the sounds whales make as they migrate through the ocean.

Advertisement

Animal behavior research can fundamental insights into the natural processes that affect ecosystems around the globe, as well as into our own human minds and behavior.

I study animal behavior – and also the research reported by scientists in my field. One of the challenges of this kind of science is making sure our own assumptions don't influence what we think we see in animal subjects. Like all people, how scientists see the world is shaped by biases and expectations, which can affect how data is recorded and reported. For instance, scientists who in a society with strict gender roles for women and might interpret things they see animals doing as reflecting those same divisions.

The scientific corrects for such mistakes over time, but scientists have quicker methods at their disposal to minimize potential observer bias. Animal behavior scientists haven't always used these methods – but that's changing. A new study confirms that, over the past decade, studies increasingly adhere to the rigorous best practices that can minimize potential biases in animal behavior research.

Black and white photo of a horse with a man and a small table between them displaying three upright cards.

Adding up?

Karl Krall/Wikimedia Commons

Advertisement

Biases and self-fulfilling prophecies

A German horse named Clever Hans is widely known in the history of animal behavior as a classic example of unconscious bias leading to a false result.

Around the turn of the 20th century, Clever Hans was purported to be able to do math. For example, in response to his owner's prompt “3 + 5,” Clever Hans would tap his hoof eight times. His owner would then reward him with his favorite vegetables. Initial observers reported that the horse's abilities were legitimate and that his owner was not being deceptive.

However, careful analysis by a young scientist named Oskar Pfungst revealed that if the horse could not see his owner, he couldn't answer correctly. So while Clever Hans was not good at math, he was incredibly good at observing his owner's subtle and unconscious cues that gave the math answers away.

In the 1960s, researchers asked human study participants to code the learning ability of rats. Participants were told their rats had been artificially selected over many generations to be either “bright” or “dull” learners. Over several weeks, the participants ran their rats through eight different learning experiments.

Advertisement

In seven out of the eight experiments, the human participants ranked the “bright” rats as being better learners than the “dull” rats when, in reality, the researchers had randomly picked rats from their breeding colony. Bias led the human participants to see what they thought they should see.

Eliminating bias

Given the clear potential for human biases to skew scientific results, textbooks on animal behavior research methods from the 1980s onward have implored researchers to verify their work using at least one of two commonsense methods.

One is making sure the researcher observing the behavior does not know if the subject from one study group or the other. For example, a researcher would measure a cricket's behavior without knowing if it came from the experimental or control group.

The other best practice is utilizing a second researcher, who has fresh eyes and no knowledge of the data, to observe the behavior and code the data. For example, while analyzing a file, I count chickadees taking seeds from a feeder 15 times. Later, a second independent observer counts the same number.

Advertisement

Yet these methods to minimize possible biases are often not employed by researchers in animal behavior, perhaps because these best practices take more time and effort.

In 2012, my colleagues and I reviewed nearly 1,000 articles published in five leading animal behavior journals between 1970 and 2010 to see how many reported these methods to minimize potential bias. Less than 10% did so. By contrast, the journal Infancy, which focuses on human infant behavior, was far more rigorous: Over 80% of its articles reported using methods to avoid bias.

It's a problem not just confined to my field. A 2015 of published articles in the sciences found that blind protocols are uncommon. It also found that studies using blind methods detected smaller differences between the key groups being observed to studies that didn't use blind methods, suggesting potential biases led to more notable results.

In the years after we published our article, it was cited regularly and we wondered if there had been any improvement in the field. So, we recently reviewed 40 articles from each of the same five journals for the year 2020.

Advertisement

We found the rate of papers that reported controlling for bias improved in all five journals, from under 10% in our 2012 article to just over 50% in our new review. These rates of reporting still lag behind the journal Infancy, however, which was 95% in 2020.

All in all, things are looking up, but the animal behavior field can still do better. Practically, with increasingly more portable and affordable audio and video recording technology, it's getting easier to carry out methods that minimize potential biases. The more the field of animal behavior sticks with these best practices, the stronger the foundation of knowledge and public trust in this science will become.The Conversation

Todd M. Freeberg, Professor and Associate Head of Psychology, University of Tennessee

This article is republished from The Conversation under a Creative Commons license. Read the original article.

Advertisement
Continue Reading

The Conversation

Boeing’s Starliner is about to launch − if successful, the test represents an important milestone for commercial spaceflight

Published

on

theconversation.com – Wendy Whitman Cobb, Professor of Strategy and Security Studies, Air – 2024-05-02 07:24:25

Boeing's Starliner spacecraft on approach to the International Space Station during an uncrewed test in 2022.

Bob Hines/NASA

Wendy Whitman Cobb, Air University

If all goes well late on May 6, 2024, NASA astronauts Butch Wilmore and Suni Williams will blast off into space on Boeing's Starliner spacecraft. Launching from the Kennedy Space Center, this last crucial test for Starliner will test out the new spacecraft and take the pair to the International Space Station for about a week.

Advertisement

Part of NASA's commercial crew program, this long-delayed mission will represent the vehicle's first crewed launch. If successful, it will give NASA – and in the future, space tourists – more options for getting to low Earth orbit.

Two people wearing blue jumpsuits hug in front of a plane.

Suni Williams, right, and Butch Wilmore, the two astronauts who will crew the Starliner test.

AP Photo/Terry Renna

From my perspective as a space policy expert, Starliner's launch represents another significant milestone in the development of the commercial space industry. But the mission's troubled history also shows just how difficult the path to space can be, even for an experienced company like Boeing.

Origins and development

Following the retirement of NASA's space shuttle in 2011, NASA invited commercial space companies to the agency transport cargo and crew to the International Space Station.

Advertisement

In 2014, NASA selected Boeing and SpaceX to build their respective crew vehicles: Starliner and Dragon.

Boeing's vehicle, Starliner, was built to carry up to seven crew members to and from low Earth orbit. For NASA missions to the International Space Station, it will carry up to four at a time, and it's designed to remain docked to the station for up to seven months. At 15 feet, the capsule where the crew will sit is slightly bigger than an Apollo command module or a SpaceX Dragon.

Boeing designed Starliner to be partially reusable to reduce the cost of getting to space. Though the Atlas V rocket it will take to space and the service module that supports the craft are both expendable, Starliner's crew capsule can be reused up to 10 times, with a six-month turnaround. Boeing has built two flightworthy Starliners to date.

A conical vehicle sitting on a flat vehicle.

The Starliner capsule in transit.

AP Photo/John Raoux

Advertisement

Starliner's development has come with setbacks. Though Boeing received US$4.2 from NASA, with $2.6 billion for SpaceX, Boeing spent more than $1.5 billion extra in developing the spacecraft.

On Starliner's first uncrewed test flight in 2019, a series of software and hardware failures prevented it from getting to its planned orbit as well as docking with the International Space Station. After testing out some of its , it landed successfully at White Sands Missile Range in New Mexico.

In 2022, after identifying and making more than 80 fixes, Starliner conducted a second uncrewed test flight. This time, the vehicle did successfully dock with the International Space Station and landed six days later in New Mexico.

The inside of a Starliner a few astronauts. Crew members first trained for the launch in a simulator.

Still, Boeing delayed the first crewed launch for Starliner from 2023 to 2024 because of additional problems. One involved Starliner's parachutes, which help to slow the vehicle as it returns to Earth. Tests found that some links in those parachute lines were weaker than expected, which could have caused them to break. A second problem was the use of flammable tape that could pose a fire hazard.

Advertisement

A major question stemming from these delays concerns why Starliner has been so difficult to develop. For one, NASA officials admitted that it did not as much oversight for Starliner as it did for SpaceX's Dragon because of the agency's familiarity with Boeing.

And Boeing has experienced several problems recently, most visibly with the safety of its airplanes. Astronaut Butch Wilmore has denied that Starliner's problems reflect these troubles.

But several of Boeing's other space activities beyond Starliner have also experienced mechanical failures and budget pressure, including the Space Launch System. This system is planned to be the main rocket for NASA's Artemis program, which plans to return humans to the Moon for the first time since the Apollo era.

Significance for NASA and commercial spaceflight

Given these difficulties, Starliner's success will be important for Boeing's future space efforts. Even if SpaceX's Dragon can successfully transport NASA astronauts to the International Space Station, the agency needs a backup. And that's where Starliner comes in.

Advertisement

Following the Challenger explosion in 1986 and the Columbia shuttle accident in 2003, NASA retired the space shuttle in 2011. The agency was left with few options to get astronauts to and from space. a second commercial crew vehicle provider means that NASA will not have to depend on one company or vehicle for space launches as it previously had to.

Perhaps more importantly, if Starliner is successful, it could compete with SpaceX. Though there's no crushing demand for space tourism right now, and Boeing has no plans to market Starliner for tourism anytime soon, competition is important in any market to down costs and increase innovation.

More such competition is likely coming. Sierra Space's Dream Chaser is planning to launch later this year to transport cargo for NASA to the International Space Station. A crewed version of the space plane is also being developed for the next round of NASA's commercial crew program. Blue Origin is working with NASA in this latest round of commercial crew contracts and developing a lunar lander for the Artemis program.

A conical white spacecraft with two rectangular solar panels in space, with the Earth in the background.

SpaceX's dragon capsule.

NASA TV via AP

Advertisement

Though SpaceX has made commercial spaceflight look relatively easy, Boeing's rocky experience with Starliner shows just how hard spaceflight continues to be, even for an experienced company.

Starliner is important not just for NASA and Boeing, but to demonstrate that more than one company can find success in the commercial space industry. A successful launch would also give NASA more confidence in the industry's ability to operations in Earth's orbit while the agency focuses on future missions to the Moon and beyond.The Conversation

Wendy Whitman Cobb, Professor of Strategy and Security Studies, Air University

This article is republished from The Conversation under a Creative Commons license. Read the original article.

Advertisement
Continue Reading

The Conversation

Brain cancer in children is notoriously hard to treat – a new mRNA cancer vaccine triggers an attack from within

Published

on

theconversation.com – Christina von Roemeling, Assistant Professor of Neurosurgery, of Florida – 2024-05-01 10:01:09

How cancer vaccines are delivered into the body influences their effectiveness.

Liuhsihsiang/iStock via Getty Images Plus

Christina von Roemeling, University of Florida and John Ligon, University of Florida

Brain cancers remain among the most challenging tumors to treat. They often don't respond to traditional treatments because many chemotherapies are unable to penetrate the protective barrier around the brain. Other treatments like radiation and surgery can with lifelong debilitating side effects.

Advertisement

As a result, brain cancer is the leading cause of cancer-related death in children. Brain tumors in children frequently do not respond to treatments developed for adults, likely due to the fact that pediatric brain cancers are not as well-studied as adult brain cancers. There is an urgent need to develop new treatments specific to children.

We developed a new messenger-RNA, or mRNA, cancer vaccine, described in newly published research, that can deliver treatments more effectively in children who have brain cancer and teach their immune systems to fight back.

Close-up of child's hand with IV line placed held by adult's hand

Cancer treatments designed for adults may not necessarily work as well in children.

Virojt Changyencham/Moment via Getty Images

How do cancer vaccines work?

The immune system is a complex network of cells, tissues and organs whose primary function is to continuously surveil the body for threats posed by foreign invaders – pathogens that tissues and make you sick. It accomplishes this by recognizing antigens, or abnormal proteins or molecules, on pathogens. T cells that recognize these antigens seek out and destroy the pathogens.

Advertisement

Your immune system also protects you from domestic threats like cancer. Over time, your cells sustain DNA damage from either internal or external stressors, leading to mutations. The proteins and molecules produced from mutated DNA look quite different from the ones cells typically produce, so your immune system can recognize them as antigens. Cancer develops when cells accumulate mutations that enable them to continue to grow and divide while simultaneously going undetected by the immune system.

In 1991, scientists identified the first tumor antigen, helping lay the framework for modern-day immunotherapy. Since then, researchers have identified many new tumor antigens, facilitating the of cancer vaccines. Broadly, cancer vaccines deliver tumor antigens into the body to teach the immune system to recognize and attack cancer cells that display those antigens. Although all cancer vaccines conceptually work very similarly, they each significantly vary in the way they are developed and the number and combination of antigens they carry.

Cancer vaccines the immune system differentiate between healthy cells and tumor cells.

One of the biggest differences among cancer vaccines is how they are created. Some vaccines use protein fragments, or peptides, of tumor antigens that are directly given to patients. Other vaccines use viruses reengineered to express cancer antigens. Even more complex are vaccines where a patient's own immune cells are collected and trained to recognize cancer antigens in a laboratory before being delivered back to the patient.

Currently, there is a lot of excitement and focus among researchers on developing mRNA-based cancer vaccines. Whereas DNA is the blueprint of which proteins to make, mRNA is a copy of the blueprint that tells cells how to build these proteins. Thus, researchers can use mRNA to create blueprint copies of potential antigens.

Advertisement

mRNA cancer vaccines

The pandemic brought significant attention to the potential of using mRNA-based vaccines to stimulate the immune system and protection against the antigens they encode for. But researchers have been investigating the use of mRNA vaccines for treating various cancers since before the pandemic.

Our team of scientists in the Brain Tumor Immunotherapy Program at University of Florida has spent the past 10 years developing and optimizing mRNA vaccines to treat brain cancer.

Cancer vaccines have significant challenges. One key hurdle is that these vaccines may not always trigger a strong enough immune response to eradicate the cancer completely. Moreover, tumors are not made up of one type of cancer cell, but rather a complex mix of cancer cells that each harbors its own unique cocktail of mutations.

Our cancer vaccine seeks to address these issues in a number of ways.

Advertisement

Diagram of lipid molecules formed a spherical shell around single-stranded mRNAs

Lipid nanoparticles can carry therapeutic mRNA into the body.

Buschmann et al. 2021/ Wikimedia Commons, CC BY-SA

First, we designed our vaccines by using the RNA of a patients' own cancer cells as a template for the mRNA inside our nanoparticles. We also packaged our cancer vaccine inside of nanoparticles made up of specialized lipids, or fat molecules. We maximized the amount of mRNA packaged within each nanoparticle by sandwiching them between lipid layers like the layers of an onion. In this way, we increase the likelihood that the mRNA molecules in our nanoparticles produce enough tumor antigens from that patient's cancer to activate an immune response.

Also, instead of injecting nanoparticles into the skin, muscle or directly into the tumor, as is commonly done for many therapeutic cancer vaccines, our mRNA nanoparticles are injected into the bloodstream. From there, they travel to organs throughout the body involved in the immune response to teach the body to fight against the cancer. By doing so, we've found that the immune system launches a near immediate and powerful response. Within six hours of receiving the vaccine, there is a significant increase in the amount of blood markers connected to immune activation.

Looking to the future

Our mRNA-based vaccines are currently undergoing early-phase clinical trials to treat real patients with brain cancer.

Advertisement

We administered our mRNA-based vaccine to four adult patients with glioblastoma who had relapsed after previous treatment. All patients survived several months longer than the expected average survival at this advanced stage of illness. We expect to treat children with a type of brain tumor called pediatric high-grade glioma by the end of the year.

Importantly, mRNA vaccines can be developed to treat any kind of cancer, including childhood brain tumors. Our Pediatric Cancer Immunotherapy Initiative focuses on developing new immune-based therapies for children afflicted with cancer. After developing an mRNA vaccine for glioma in chidren, we will expand to treat other kinds of pediatric brain cancers like medulloblastoma and potentially treat other kinds of cancers like skin cancer and bone cancer.

We are hopeful that mRNA-based vaccines may to more children being cured of their brain tumors.The Conversation

Christina von Roemeling, Assistant Professor of Neurosurgery, University of Florida and John Ligon, Assistant Professor of Hematology, University of Florida

This article is republished from The Conversation under a Creative Commons license. Read the original article.

Advertisement
Continue Reading

News from the South

Trending