Using Code Interpreter

A deeper dive into the Swiss Army knife of GPT plug-ins

JS game made by Code Interpreter.

Another Look at OpenAI’s Code Interpreter Plug-In

In early May, OpenAI released their most powerful GPT-4 Plug-In yet: Code Interpreter. A flurry of excitement followed, with early users sharing stories of the innovative things they could do with it. We shared that excitement in our article “OpenAI's chatGPT code interpreter plug-in changes everything!”

Last week, OpenAI general availability for Code Interpreter, which means anyone subscribed to OpenAI ChatGPT plus ($20 / month), now can use this plug-in; it makes the upgrade worth it. We will dig into it a bit more and show a bit of what it can do.

Capabilities of Code Interpreter

The main point of Code Interpreter is that GPT-4 is capable of interpreting a prompt and writing Python, and the code interpreter plug-in can run the code. That combination can be considered a design pattern to go from natural language request to the output of a coding operation.

The result is taking the power of an LLM and forging with it a tool that has countless applications, which opens up a world of possible applications. What we said in early May on this:

This hype is driven by the how shockingly easy it is now to call upon chaptGPT to perform sophisticated tasks of data analysis, data visualization, file conversion, utility coding, and more. It’s like having a data scientist and web developer on your utility belt. Amazing!

Code Interpreter already has many capabilities thanks to its ability to accept just about any file type as an input. As people grow more accustomed to this tool, other niche uses could be determined.

Reading, understanding and writing code

GPT-4 was trained on vast amounts of code along with vast amounts of text, so its facility with both creates remarkable abilities. It is good at generating code for many Python programs or functions, thanks to being trained on thousands of examples.

It further excels at explaining code. Besides having it simply write us code for short algorithms, it can explain what it did in English. Given code as input, it can describe the algorithm used, make code corrections and give suggestions about the program. This is like having a programming language intern or even a programming language tutor.

Here are some results on tying out a simple programming task, asking Code Interpreter to generate python code to apply Caesar Cipher (equal integer shifting of the characters) to an ASCII text input:

Code Interpreter also explains step-by-step how the function works:

Also notice that it caught a flaw from the original input, the lack of a wraparound for the ASCII table. After asking it to add in the wraparound, it created our final function:

Code Interpreter offers inline running of the code it produces, but since this is only a function, it needs to have inputs specified when run online, and be given a source of input when run locally. It can also be copied and run locally.

Debugging and improving code: We edited the python script to have a random offset and string input, as well as several errors, including semantic, formatting, arithmetic, and logic. Code Interpreter catches them all. It then assesses how to change them in order to achieve the desired result. It also correctly identifies the code as the Caesar Cipher, despite the file name not including it and the original prompts above not mentioning it.

Code Interpreter does have a little trouble with running the code since it has no input window, but quickly assesses the necessity of a temporary test value, and chooses “Hello World!”. Most impressive that it fixes the errors, checks it via running, and reports its results.

Corrected Code:

Data Analytics with Code Interpreter

Code Interpreter is especially useful for data analysis because it allows users unfamiliar with programming to use natural language queries to harness the power of data analysis algorithms on imported data.

Code Interpreter has pre-loaded python packages for this, such as numpy, pandas, matplotlib, scipy, sklearn, etc. This give it the ability to accept and convert various data file types; access a number of python coded tasks to clean and pre-process data; run various analyses on it; and then produce graphics for it. The power of the python under the hood is the power of flexibility.

Whether that’s graphing trends from data or cleaning inputs, Code Interpreter can automate a lot of the data processing and data analysis flow.

For example, we can feed it several thousand lines of data in a CSV and ask for visualization plots. Code Interpreter puts together algorithms to read this data in moments, and before you know it has a plot finished.

Code Interpreter for Document and Multimedia Processing

Code Interpreter is also able to accept images and audio and operate on them, thanks to having python packages to read and process video files. Code Interpreter proves itself a versatile tool in splicing and editing videos, extracting audio from videos, as well as image editing, unexpected ways.

However, the caveat is that it may not be as powerful as simply using software tools that do specialized tasks directly. This AI is leveraging open source code and python packages, and so the power is more about getting access to them easily, while accepting the limitations that Code Interpreter only has a limited number of python libraries to draw from.

Sometimes it claims not to have a power it really has and needs a push. I asked it to read an image and output the text in it, and it claimed it couldn’t do it, giving me python code and library to try locally. But with coaxing by saying “Your current environment does support it so give it a try,” it attempted to to it, and it worked.

In the worst case, it will be unable to run the code, but can easily generate the code that you can run locally. For example, we asked about converting a video file into a transcript, it gave the recipe and the python code, but could not run it due to lack of dependencies.

Using Code Interpreter for Websites

More than just writing quick python functions, Code Interpreter can take on many types of programs, including creating small web applications. We were able to build a simple one in minutes.

Code Interpreter created correct code for this simple website, which runs locally and keeps track of how many times you press a button. Although this is a very rudimentary example, it displays Code Interpreter’s wide range of abilities.

A more ambitious website request prompt led to the response:

“Creating a complete, robust Django application with all the features you mentioned is quite extensive and beyond the scope of this platform, but I can certainly help you get started with some basic code and explain the structure and components you'd need.”

It then gave a very detailed outline for the Django website application, albeit a rather standard framework. One might conclude from this that it is very helpful for the novice and to get started, but the jury is out on whether it can be helpful in adding value to more advanced users asking niche questions.

Limitations

Code Interpreter is a most powerful and general plug-in, but it has many limitations:

• The sandbox: Running code is done in the interpreter engine, a container that for safety reasons is blocked off from the internet. This is great for small coding or data analysis, but without connection to the internet, and ability to upload libraries, it lacks some extensibility and power it might otherwise have.

• Bring your own data: Related to the sandbox is that to analyze or use data, you have to upload and download. You can’t upload files that are too large to the interface; the limit is 100MB.

• Python only: While GPT-4 can write code in many programming languages, Code Interpreter supports running Python only, which means execution tasks are limited to what Python can do. This may be inefficient for some tasks that are better suited to a different language.

We also wonder what about it limits on more complex custom applications. We were easily able to generate standard application examples, but it stumbled on more custom niche requests. How helpful can it be on tasks for more advanced users?

Conclusion

The Code Interpreter plugin is extremely versatile and powerful, as it leverages Python's versatile execution capabilities to extend GPT-4's AI power. Despite the above-mentioned limitations what Code Interpreter can do is so useful and broad, this plug-in could be the ‘killer app’ for AI, even more than ChatGPT was. This is useful.

While useful, it could be even more so. The plugin could improve and benefit from internet access, support for more languages, and integration within code development environments.

Also, imagine the versatility if the LLM had complete control of its execution environment, the docker-style container serving as the execution 'sandbox'. Direct control could pave the way for more advanced applications, including running AI models or other programs within the container.

As the field evolves, similar plug-ins to Code Interpreter are emerging from competitors. Google Bard connects with the Replit code environment. Coding co-pilots might also evolve to enable similar capabilities to Code Interpreter.

As with all things AI, it will keep getting better, and so however it evolves, we the users of these tools will win.