CodeT5: The Code-aware Encoder-Decoder based Pre-trained Programming Language Models

Yue Wang · #code-intelligence

TL; DR: Introducing CodeT5 --- the first code-aware, encoder-decoder-based pre-trained programming language model, which enables a wide range of code intelligence applications including code understanding and generation tasks. CodeT5 achieves state-of-the-art performance on 14 sub-tasks in the CodeXGLUE code intelligence benchmark.CodeT5 for code-related understanding and generation tasksGiven the goal

Learning without Labels

Michael Sollami · #deeplearning

With data rapidly being generated by millions of people, it's not feasible to label all of it. Learn about the recent advancements in ML for how to train vision models with unlabelled data using self-supervised learning.

Salesforce Research at ICLR 2021

Mia Ferrer ·

This year marks the 9th annual conference on International Conference on Learning Representations (ICLR) taking place in a fully virtual format from May 4th through May 8th, 2021. ICLR is a premier academic conference in the field of representation learning, generally referred to as deep learning or feature learning. ICLR

When are Neural Networks more powerful than Neural Tangent Kernels?

Yu Bai · #deep learning theory

The empirical success of deep learning has posed significant challenges to machine learning theory: Why can we efficiently train neural networks with gradient descent despite its highly non-convex optimization landscape? Why do over-parametrized networks generalize well? The recently proposed Neural Tangent Kernel (NTK) theory offers a powerful framework for understanding

Applying AI Ethics Research in Practice

Kathy Baxter · #ethics

Summary from FAccT 2020 CRAFT SessionAI Ethics practitioners in industry look to researchers for insights on how to best identify and mitigate harmful bias in their organization’s AI solutions and create more fair or equitable outcomes. However, it can be a challenge to apply those research insights in practice.

Salesforce Research at NeurIPS 2020

Denna Mafie ·

This year marks the 34th annual conference on Neural Information Processing Systems (NeurIPS) reimagined for the first time ever in a fully virtual format. NeurIPS is a leading conference in the area of machine learning and neural information processing systems in their biological, technological, mathematical, and theoretical aspects. Neural information

CoMatch: Advancing Semi-supervised Learning with Contrastive Graph Regularization

Junnan Li ·

TL; DR: We propose a new semi-supervised learning method which achieves state-of-the-art performance by learning jointly-evolved class probabilities and image representations.What are the existing semi-supervised learning methods?Semi-supervised learning aims to leverage few labeled data and a large amount of unlabeled data. As a long-standing and widely-studied topic in

Salesforce Research at EMNLP 2020

Denna Mafie · #research

This year marks the 24th annual Empirical Methods in Natural Language Processing (EMNLP) conference reimagined for the first time ever in a fully virtual format. EMNLP is a leading conference in the area of Natural Language Processing covering a broad spectrum of diverse research areas that are concerned with computational

A Language Detector for Identifying Machine-Generated Text

Yoav Schlesinger ·

In recent years, the natural language processing (NLP) community has seen the development of increasingly powerful language models [1, 2], capable of generating textual output that is indistinguishable from human-written text. This includes our own model called CTRL [3] (Conditional Transformer Language Model) for controllable generation. To prevent misuse or

The First Simulation Card for Ethical AI Simulations

Stephan Zheng ·

We recently released Foundation, an open-source framework to build economic simulations. Foundation has been designed with flexibility and AI research in mind, and can be modified by anyone. AI simulations offer researchers the power to generate data and evaluate outcomes of virtual economies that capture a part of the real

Model Cards for AI Model Transparency

Yoav Schlesinger · #ethics

At Salesforce, we take seriously our mission to create and deliver AI technology that is responsible, accountable, transparent, empowering, and inclusive. These principles ensure that our AI is safe, ethical, and engenders trust.

Theory-Inspired Network Architecture Search

Pan Zhou ·

TL;DR: We theoretically analyze the differential architecture search (DARTS) for understanding the role and impact of skip connections, which inspires a new method for Neural Architecture Search (NAS) using group-structured sparse gates and path-depth-wise regularization to overcome the limitation of existing NAS methods for AutoML. In our work [1]

How Salesforce Infuses Ethics into its AI

Katherine Siu · #artificial intelligence

For all the good that AI can bring, responsible tech companies understand they must recognize, prepare for, and mitigate the potential unintended, harmful effects. That’s why Salesforce sees ethics as foundational to AI — and why we’re sharing a closer look at how we infuse an ethical process into

The AI Economist: Join the Moonshot

Stephan Zheng ·

We are launching an open source collaborative project to build an AI Economist that can be used to guide policy making in the real world. We invite you to join us in our mission to help improve the world with AI and economics.

Salesforce Research at ACL 2020

Audrey Cook ·

The 58th Association for Computational Linguistics (ACL) Conference kicked off this week and runs from Sunday, Jul 5 to Friday, Jul 10 in a fully virtual format. ACL is the premier conference of the field of computational linguistics, covering a broad spectrum of diverse research areas that are concerned with

ERASER: A Benchmark to Evaluate Rationalized NLP Models

Nazneen Rajani · #research

Many NLP applications today deploy state-of-the-art deep neural networks that are essentially black-boxes. One of the goals of Explainable AI (XAI) is to have AI models reveal why and how they make their predictions so that these predictions are interpretable by a human. But work in this direction has been