GitHub Co-Pilot Test Flight

GitHub Co-Pilot

A service provided by github to suggest code snippets based on editor context. Powered by openAI model which was trained over a billion lines of public code hosted on GitHub. The co-pilot website has a lot of example use cases which are so accurate they seem unreal. Beyond those examples, running it myself with funny comments and function signature prompts actually gives pretty usable code-snippets.

Setting Up

With VSCode integration, setting-up co-pilot is as easy as it gets. Once the extension is installed it is really intuitive and feels like any other IDE suggestion tool but a lot smarter.

Not a Search Engine

It is supposed to be a suggestion engine instead of search. You can use google, stack overflow, github search to find a solution to your problem and a code snippet for it, but co-pilot is supposed to be smarter than that and provide you suggestions instead of a copy paste of your search result. At least what it is supposed to be. But the question is what will happen if I provide a very specific context which will force the co-pilot to produce a suggestion which is nothing but a search result.

vs SearchEngine

I often use DDG to search for code snippet and in most cases I get a parsed response from stackoverflow neatly displayed in a box right next to search result.

Let’s try few example

Email Validation

DDG

Search Query: email regex javascript

Result:

function validateEmail(email) {
  const re =
    /^(([^<>()\[\]\\.,;:\s@"]+(\.[^<>()\[\]\\.,;:\s@"]+)*)|(".+"))@((\[[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\])|(([a-zA-Z\-0-9]+\.)+[a-zA-Z]{2,}))$/;
  return re.test(String(email).toLowerCase());
}

copilot

Prompt:

// email regex
function validateEmail(email)

Suggestion

// email regex
function validateEmail(email) {
  var re =
    /^(([^<>()[\]\\.,;:\s@\"]+(\.[^<>()[\]\\.,;:\s@\"]+)*)|(\".+\"))@((\[[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\])|(([a-zA-Z\-0-9]+\.)+[a-zA-Z]{2,}))$/;
  return re.test(email);
}

Array Max

DDG

Search Query: find max in array javascript

Result:

Math.max ( [ value1 [ , value2 [ , … ] ] ] )

copilot

Prompt:

// find max in array

Suggestion:

// find max in array
function max(arr) {
  return Math.max.apply(null, arr);
}

Leetcode problems

DDG

Search Query: leetcode reverse Nodes in k-group Result:

Link to the problem & some related searches but no code

copilot

Prompt:

# leetcode reverse Nodes in k-group

Then another one once the comment section was over

class Solution:

And copilot gave the whole solution instantly

Conclusion

For this test the results produced by copilot were a lot more useful and production ready. Plus I didn’t have to leave the IDE.