Thursday, February 2, 2023
  • Login
CEO North America
  • Home
  • Business
    • Entrepreneur
    • Industry
    • Innovation
    • Management & Leadership
  • CEO Interviews
  • CEO Life
    • Art & Culture
    • Food
    • Health
    • Travel
    • Environment
  • Opinion
  • News
  • Multimedia
No Result
View All Result
  • Home
  • Business
    • Entrepreneur
    • Industry
    • Innovation
    • Management & Leadership
  • CEO Interviews
  • CEO Life
    • Art & Culture
    • Food
    • Health
    • Travel
    • Environment
  • Opinion
  • News
  • Multimedia
No Result
View All Result
CEO North America
No Result
View All Result

A decade in deep learning, and what’s next

in Innovation
Amerisourcebergen begins investing in j. P. Morgan’s empower share class to support minority-owned & diverse-led financial institutions
Share on FacebookShare on Twitter

I was first introduced to neural networks — computer systems that roughly imitate how biological brains accomplish tasks — as an undergrad in 1990. I did my senior thesis on using parallel computation to train neural networks. In those early days, I thought if we could 32X more compute power (using 32 processors at the time!), we could get neural networks to do impressive things. I was way off. It turns out we would need about 1 million times as much computational power before neural networks could scale to real-world problems.

A decade later, as an early employee at Google, I became reacquainted with machine learning when the company was still just a startup. In 2001 we used a simpler version of machine learning, statistical ML, to detect spam and suggest better spellings for people’s web searches. But it would be another decade before we had enough computing power to revive a more computationally-intensive machine learning approach called deep learning. Deep learning uses neural networks with multiple layers (thus the “deep”), so it can learn not just simple statistical patterns, but can learn subtler patterns of patterns — such as what’s in an image or what word was spoken in some audio. One of our first publications in 2012 was on a system that could find patterns among millions of frames from YouTube videos. That meant, of course, that it learned to recognize cats.

To get to the helpful features you use every day — searchable photo albums, suggestions on email replies, language translation, flood alerts, and so on — we needed to make years of breakthroughs on top of breakthroughs, tapping into the best of Google Research in collaboration with the broader research community. Let me give you just a couple examples of how we’ve done this.

A big moment for image recognition

In 2012, a paper wowed the research world for making a huge jump in accuracy on image recognition using deep neural networks, leading to a series of rapid advances by researchers outside and within Google. Further advances led to applications like Google Photos in 2015, letting you search photos by what’s in them. We then developed other deep learning models to help you find addresses in Google Maps, make sense of videos on YouTube, and explore the world around you using Google Lens. Beyond our products, we applied these approaches to health-related problems, such as detecting diabetic retinopathy in 2016, and then cancerous cells in 2017, and breast cancer in 2020. Better understanding of aerial imagery through deep learning let us launch flood forecasting in 2018, now expanded to cover more than 360 million people in 2021. It’s been encouraging to see how helpful these advances in image recognition have been.

Similarly, we’ve used deep learning to accelerate language understanding. With sequence-to-sequence learning in 2014, we began looking at how to understand strings of text using deep learning. This led to neural machine translation in Google Translate in 2016, which was a massive leap in quality, particularly for less prevalent languages. We developed neural language models further for Smart Reply in Gmail in 2017, which made it easier and faster for you to knock through your email, especially on mobile. That same year, Google invented Transformers, leading to BERT in 2018, then T5, and in 2021 MUM, which lets you ask Google much more nuanced questions. And with “sparse” models like GShard, we can dramatically improve on tasks like translation while using less energy.

We’ve driven a similar arc in understanding speech. In 2012, Google used deep neural networks to make major improvements to speech recognition on Android. We kept advancing the state of the art with higher-quality, faster, more efficient speech recognition systems. By 2019, we were able to put the entire neural network on-device so you could get accurate speech recognition even without a connection. And in 2021, we launched Live Translate on the Pixel 6 phone, letting you speak and be translated in 48 languages — all on-device, while you’re traveling with no Internet.

More invention ahead

As our research goes forward, we’re balancing more immediately applied research with more exploratory fundamental research. So we’re looking at how, for example, AI can aid scientific discovery, with a project like mapping the brain of a fly, which could one day help better understand and treat mental illness in people. We’re also pursuing quantum computing, which will likely take a decade or longer to reach wide-scale applications. This is why we publish nearly 1000 papers a year, including around 200 related to responsible AI, and we’ve given over 6500 grants to external researchers over the past decade and a half.

Looking ahead from 2021 to 2031, I’m excited about the next-generation AI systems we can build, and how much more helpful they’ll be. We’re planting the seeds today with new architectures like Pathways, with more to come.

(Courtesy Google/by Jeff Dean)

Tags: DecadeDeep learningGoogle

Related Posts

Business benefits of a customer self-service strategy
Business

Business benefits of a customer self-service strategy

When retail buyers panic, a. I. Keeps calm
Business

When Retail Buyers Panic, A.I. Keeps Calm

A simpler path to better computer vision
Business

A Simpler Path to Better Computer Vision

Trust, risk, and opportunity: overseeing a comprehensive data and privacy strategy
Innovation

Trust, Risk, and Opportunity: Overseeing a Comprehensive Data and Privacy Strategy

Engineers solve a mystery on the path to smaller, lighter batteries
Innovation

Engineers Solve a Mystery on the Path to Smaller, Lighter Batteries

Adopting an ecosystem-first mindset in software
Business

Adopting an Ecosystem-First Mindset in Software

Deep tech powers new net-zero business models
Business

Deep Tech Powers New Net-Zero Business Models

Mit engineers develop a low-cost terahertz camera
Business

MIT Engineers Develop a Low-Cost Terahertz Camera

Using machine learning to identify undiagnosable cancers
Innovation

Using Machine Learning to Identify Undiagnosable Cancers

Show me the digital value
Innovation

Show Me the Digital Value

No Result
View All Result

Recent Posts

  • Tinder owner to lay off 8% of its staff as growth falters
  • Biden moves to slash U.S. credit card fees, app charges
  • New York City Votes To “Skip The Stuff” In Take-Out Orders
  • How to Test If Your Interview Process Is a Nightmare
  • The Next Era of Work Will Be About Skills, Not Pedigree

Recent Comments

    Archives

    Categories

    • Art & Culture
    • Business
    • CEO Interviews
    • CEO Life
    • Editor´s Choice
    • Entrepreneur
    • Environment
    • Food
    • Health
    • Highlights
    • Industry
    • Innovation
    • Issues
    • Management & Leadership
    • Multimedia
    • News
    • Opinion
    • PrimeZone
    • Printed Version
    • Travel
    • Uncategorized

    Meta

    • Log in
    • Entries feed
    • Comments feed
    • WordPress.org

    • CONTACT
    • GENERAL ENQUIRIES
    • ADVERTISING
    • MEDIA KIT
    • DIRECTORY
    • TERMS AND CONDITIONS

    Editorials – stuart.james@ceo-na.com

    Editor-In-Chief – caroline.sposto@ceo-na.com

    Editorials – editorials@ceo-na.com

    Advertising – media@ceo-na.com

    NEW YORK

    110 Wall St.,
    3rd Floor
    New York, NY.
    10005
    USA
    +1 212 432 5800

     

    MEXICO CITY

    Paseo de la Reforma 296,
    Floor 38
    Mexico City
    06600
    MEXICO

    • CONTACT
    • GENERAL ENQUIRIES
    • ADVERTISING
    • MEDIA KIT
    • DIRECTORY
    • TERMS AND CONDITIONS

    Editorials –
    stuart.james@ceo-na.com

    Editor-In-Chief –

    caroline.sposto@ceo-na.com

    Editorials – editorials@ceo-na.com

    Advertising –
    media@ceo-na.com

    NEW YORK

    110 Wall St.,
    3rd Floor
    New York, NY.
    10005
    USA
    +1 212 432 5800

    MEXICO CITY

    Paseo de la Reforma 296,
    Floor 38
    Mexico City
    06600
    MEXICO

    CEO North America © 2022 - Sitemap

    No Result
    View All Result
    • Home
    • Business
      • Entrepreneur
      • Industry
      • Innovation
      • Management & Leadership
    • CEO Interviews
    • CEO Life
      • Art & Culture
      • Food
      • Health
      • Travel
      • Environment
    • Opinion
    • News
    • Multimedia

    © 2023 JNews - Premium WordPress news & magazine theme by Jegtheme.

    Welcome Back!

    Login to your account below

    Forgotten Password?

    Retrieve your password

    Please enter your username or email address to reset your password.

    Log In