BLOG
BLOG
  • Design
  • Data
  • Discernment

We believe in AI and every day we innovate to make it better than yesterday. We believe in helping others to benefit from the wonders of AI and also in extending a hand to guide them to step their journey to adapt with future.

Know more

Our solutions in action for customers

DOWNLOAD

Featured Post

Neobank Vs Digital Bank: Everything you need to know about the future of banking

There is a new buzz in the tech world every day, usually about something that would simplify tasks even more than today. Neobank is one such tech advancement that has been gaining traction in the era of digital banking.

Know More
How to build a metaverse virtual world?

The term metaverse has been blowing up the internet the past few months. The news about Facebook being renamed as Meta was mixedly received with skepticism and excitement (mostly from the Matrix and Black Mirror fans). Experts define it as a whole new meaning to connecting virtually that could potentially replace its equivalent in the physical world. […]

Know More
Top 10 Doubts And Questions On NFTs Answered

NFTs are certainly the most trending tech topic these days. Since this technology enables non-fungible assets to be traded in the digital medium—artists, memorabilia collectors, etc have embraced it to build newer business models and marketplaces for NFTs. If you haven’t yet understood the concept of NFTs, the video embedded below can be helpful. Recently, […]

Know More
What is Initial Dex Offering or IDO? The benefits of IDO fundraising

Initial DEX offering or IDO is a new fundraising model that offers better liquidity of crypto assets, faster, open, and fair trading. IDO model is the successor of fundraising models such as ICO, STO, and IEO.  Before we learn about IDO crypto, let’s brush up on our knowledge of the fundraising concept via token sales. […]

Know More
Business Intelligence & Data Analytics in Retail Industry

The traditional data analytics in retail industry is experiencing a radical shift as it prepares to deliver more intuitive demand data of the consumers. The rise of online shopping may have a major impact on the retail stores but the brick-and-mortar sales aren’t going anywhere soon. According to Euromonitor International, it is projected that 83% […]

Know More
Artificial Intelligence For Enhancing Business Security

If you are a business owner, you already know the importance of business security. Most of the businesses are just one ‘security mishap’ away from a temporary or a total failure. Security mishaps come in different sizes and shapes, such as the occurrence of fire or thefts happening inside your business premises.  All these mishaps […]

Know More

MENU

  • Visit Accubits Website.
  • Artificial Intelligence
  • Blockchain
  • Entertainment
  • Fintech
  • General

Getting started with GPT-3 model by OpenAI – The largest AI language model ever created

  • mm
    by Raoof Naushad on Tue Aug 11

Generative Pre-trained Transformer 3, more commonly known as GPT-3 is an autoregressive language model that was created by OpenAI. It is the largest language model ever created till date and has been trained on an estimated 45 terabytes of text data, run through 175 billion parameters! The models have utilized a massive amount of data from the internet, which gives them the power to generate human-like text.

The third version of the GPT model (GPT-3) created a lot of hype in the developer community. People have been posting tweets on several awesome applications that they built using GPT-3 API. The model is in the private beta version, but the API is available upon request. A good example of an application that was built through this model is a layout generator, where you just describe any layout you want, and it generates the JSX code for you.

This is mind blowing.

With GPT-3, I built a layout generator where you just describe any layout you want, and it generates the JSX code for you.

W H A T pic.twitter.com/w8JkrZO4lk

— Sharif Shameem (@sharifshameem) July 13, 2020

There are many other examples where developers created applications that convert natural language instruction to SQL queries, HTML, poem writing, content writing and many more. The most amazing part is that there is no need of any fine tuning or training the model. Developers only have to send some sample requests to the API.

how to use gpt-3

Several methods to evaluate the performance of GPT-3 were used. A few results that were obtained demonstrated that, like many other AI models, GPT-3 also lacks common sense and can also be fooled to generate text that is incredibly biased.

GPT-3 Model

GPT-3 is great milestone in the artificial intelligence community, but the hype for GPT-3 is way too high. OpenAI’s co-founder Sam Altman commented

The GPT-3 hype is way too much. It’s impressive (thanks for the nice compliments!) but it still has serious weaknesses and sometimes makes very silly mistakes. AI is going to change the world, but GPT-3 is just a very early glimpse. We have a lot still to figure out

GPT-3 Machine Learning Model

GPT-3 is trained on a massive dataset that covered almost the entire web with 500B tokens and 175 billion parameters. Compared to its previous version, it is 100x larger as well. It is a deep neural network model for language generation that is trained in such a way that it checks for the probability of a word to exist in a sentence. For example, in the sentence “My father was a queen” , the word queen in this context is less probable than king in “My father was a king”. Thus by training this big neural network with almost all sorts of data from common crawl and Wikipedia, OpenAI made the GPT-3 model better than other models.

Julia Vs Python

Benchmarking and performance analysis of Julia Vs Python

Read More

GPT-3 is actually not a novel architecture. It also uses transformers neural networks like other popular language models like BERT and GPT-2. It performs much better than its predecessors, as it has been trained on a large amount of data, with extremely large model architecture and 175 billion parameters. As I said before, with this many parameters and lots of data, GPT-3 can do any NLP task like summarization, creating codes from text, writing poem etc. without even training or fine tuning. This is what makes GPT-3 so special. The model is not yet released but an API is available upon request.

GPT-3 model by OpenAI

Parameters comparison of GPT-3

GPT-3 when evaluated using zero shot and one shot learning shows promising performance and with few shot training, it occassionaly surpassed state of art models in the respective NLP tasks.

Getting started with GPT-3 model

Following are the steps involved in getting started with GPT-3.

  1. Get API token from OpenAI
  2. Clone the repo
  3. install openai
  4. Import modules, setup API token
  5. Add examples
  6. Submiting input

Get the API token from OpenAI

Open AI is offering free access to the API through mid-August for private beta. They’re also kicking off an academic access program to let researchers build and experiment with the API. They’re going to start with an initial set of academic researchers and collaborators who will gain free access to the API. To get the API, GO HERE and request for an API and hopefully OpenAPI will get back to you with the access token.

How to download or install GPT-3

Clone repository — Download the gpt.py file from this repository and save it in your local machine. Thanks to Shreyashankar for her amazing repository.

Install OpenAI

pip install openai

pip install openai

Import modules and setup API token

Here, we imported the required libraries. Then we setup the api_key from the access token we get from OpenAI. Now with the gpt file we downloaded before we imported two modules GPT which is used to initialize the API with parameters and Example is used to prime the data with example types.

import json
import openai
from gpt import GPT
from gpt import Example

openai.api_key = data["API_KEY"]


with open('GPT_SECRET_KEY.json') as f:
    data = json.load(f)

gpt = GPT(engine="davinci",
          temperature=0.5,
          max_tokens=100)

Add examples

Here we are trying to convert natural language to SQL with the help of very few examples.

gpt.add_example(Example('Fetch unique values of DEPARTMENT from Worker table.', 
                        'Select distinct DEPARTMENT from Worker;'))

gpt.add_example(Example('Print the first three characters of FIRST_NAME from Worker table.', 
                        'Select substring(FIRST_NAME,1,3) from Worker;'))

Submit input and get ouput.

Here we sent an input as ordinary text with submit_request and get the response.

## Example - 1
prompt = "Display the lowest salary from the Worker table."
output = gpt.submit_request(prompt)
print(output.choices[0].text)

## Example 2
prompt = "Tell me the count of employees working in the department HR."
print(gpt.get_top_reply(prompt))

Done ! Awesome right?

As I already mentioned, GPT-3 scraped almost every text data on the internet. This enabled the researches to identify how different sentiments like sexism and racism play a role in real world conversation. For example, the word ‘man’ can be found to have close relationships with words like strong, brave etc. since they co-occured more with these words when compared with the word ‘woman’.

Facetagging with Elasticsearch

How to do face tagging and clustering in videos with Elasticsearch

Read More

Despite some limitations, the GPT-3 model is one of the best models for NLP tasks. Every new piece of technology can help further mankind, and this model is just one extra step in the right direction.

Related Posts

Drive into stories worth discussing

  • mm
    How Machine Learning Powered Demand Forecasting Can Help Your Business?
    Mahadevan P V
  • mm
    Explainable AI: Why It Matters?
    Adarsh M S
  • mm
    Driving Business Innovations in the Post-digital Era with DARQ
    Aharsh MS
  • mm
    Ethical AI and Business Challenges: How to tackle biases in Artificial Intelligence
    Charush Nair

ASK AUTHOR

mm
Raoof Naushad

A technology enthusiast with an urge to explore into vast areas of advancing technologies. Experienced in domains like Computer ... Read more

Ask A Question
Error
Cancel
Send

Categories

View articles by categories

  • Artificial Intelligence
  • Blockchain
  • Entertainment
  • Fintech
  • General
  • Healthcare
  • Healthcare
  • ICOs
  • Industries
  • Insurance
  • Metaverse
  • Real Estate
  • Retail
  • Success Stories
  • Team Culture & Values

Resources

Complete ICO Checklist

Subscribe now to get our latest posts

  • facebook
  • linkedin
  • twitter
  • youtube
All Rights Reserved. Accubits Technologies Inc