Continuing the #python #CoPilot example, one can make it work by upping their prompting game. * In addition to the description of the algorithm, give the desired input and output. * It immediately suggests to define a class for intervals, followed by a line sweep over the *sorted* intervals. * It can generate some (sorted) test cases after prompting. *Surprisingly it had some issues with printing the results (for whatever reason, it could not generate the unpack-print loop, so I just did it
If one calls the code thus generated on a list of sorted intervals, the output is as expected, i.e. there are 2 groups of overlapping intervals, First Group: [1,3] , [2,4] Second Group: [5,7] , [6,8]
It works with an input of unsorted intervals (as expected). There are various tests that are not being done by the function e.g. for intervals of zero or negative lengths, duplicate intervals etc. The code's correctness is thus predicated on promises about the input.
The amount of effort to come up with a #Githubcopilot prompt that generates a functioning 1st prototype was substantially higher in #python than #perl for *this specific example*. Is this a general feature? I have maintained (and indeed this is the reason I came back to #perl) that it is a much better language for high level reasoning when coding, with a much better yield-to-character-typing ratio. Is this the reason #GenerativeAI was able to deliver with less effort? More exploration to do!!
Perl feels to me like thinking in sentences, which makes sense given that Larry Wall modeled it on patterns in natural language. I wonder what Perl code bases Copilot was trained on?
I wonder about this, too. If the #copilot was only trained using #github , then the #perl code base is *very small* ie it was used in ~0.6% of projects in 2014 and 0.3% in 2022. If these findings, i.e. #copilot can deliver better solutions in @Perl vs. other languages, generalize, then we may see a language boost and a better job market for #perl programmers. The latter would be a result of the need to optimize the initial solutions as @mjgardner did for the #perl example over here.
So the • coached (“#PromptEngineering”) • license-washing (training data illegally pilfered and stripped of provenance) • stochastic parrots (#LargeLanguageModels) need a senior #developer to help them finish the job too?
It seems like the only things “#AI” bring to the table are the shoddy #ethics of its builders and backers.
Yeap, notwithstanding the shoddy ethics, I don't think it can work on its own. At most, it will cannibalize entry level jobs (similar to what happened to human computers when digital computers came to life in the 1950s).
But assuming it takes off, the real fun will take place a few years down the road, when most of github will be composed of uncurated stochastic parrot output. I hope none of the resulting SW ends up inside a medical device
There is another dimension that should also be considered here, that of productivity. Getting the prompting right for #largelanguagemodel to generate a chunk of code that works & testing said code, may end up taking more time than actually doing the deed by hand. It makes more sense (IMHO as a hobbyist) to have good metadata about functionality for software libraries so that they can be located & reused, with #metaprogramming generates the boilerplate.
Then again, the culture has been infected so deeply with the “property is theft” anti-concept. So people get away with it… until the actual creators upstream go on strike.
We are evaluating CoPilot at work. Only senior devs have access, because the code it generates is often wrong in subtle ways that could trip up a less experienced dev. I find that it does a great job generating comments, tests, adding log messages -- boilerplate. But again, often slightly wrong (after all, it can't think, it's just patterns).
Ethics aside, it has been at best slightly helpful, in the same way that autocompletion is helpful.
With autocomplete you’re only typing a word at a time, you check as you go, and you still decide whether to hit that “send” button.
The “#AI” dream for business software development is to remove all those human checks. The business wants to fire all their mercurial developers demanding precise requirements in favor of a black box that absolves them of guilt for remixing the code it already stole from others.
Or some sort of advanced completion, that uses a natural lang prompt to say, "I need this piece of code to do XYZ" here and it is copied from a database. We have dot phrases in electronic health records that put boilerplate stuff that we then edit. But it does take a substantial amount of configuration to do something like this
Did you guys see the #FAFO (Eff Around, Find Out) when a mental health services company replaced its human councellors with #AI ? It was beautiful to behold how the disaster unfolded (in ~ 48hrs)
I agree, the best feature for me would be a programmer's assistant that provides smart, context-aware references. For example: autocomplete the headers for the AWS classic load balancer logs so I don't have to pull up the docs.
could the training data be biased? Python is a much more popular language these days, which i believe means a lot more beginners developers will use it and thus produce low quality code, while people doing Perl today are probably very invested in the language, and thus much more expert at it?
Could be - though I can't locate line sweep implementations in #perl online (though could find for #cplusplus , #Python and #java). One can further postulate that since a larger (vs #python) chunk of #perl is not oo , the #LLM was more likely to generate this code with less prompting. Or that #perl can go a long way before needing objects (sorry @ovid ). Would be nice to have a benchmark set of problems to compare #copilot 's performance across languages.
No worries about the "objects" comments. While I love OOP, I continually stress to people that it's not a silver bullet and, in fact, can often make software worse when it's not understood, or when people use it in inappropriate circumstances where procedural or declarative solutions are better.
Christos Argyropoulos MD, PhD
in reply to Christos Argyropoulos MD, PhD • • •If one calls the code thus generated on a list of sorted intervals, the output is as expected, i.e. there are 2 groups of overlapping intervals,
First Group: [1,3] , [2,4]
Second Group: [5,7] , [6,8]
It works with an input of unsorted intervals (as expected). There are various tests that are not being done by the function e.g. for intervals of zero or negative lengths, duplicate intervals etc. The code's correctness is thus predicated on promises about the input.
#LLM #python #Copilot
Christos Argyropoulos MD, PhD
in reply to Christos Argyropoulos MD, PhD • • •Is this a general feature?
I have maintained (and indeed this is the reason I came back to #perl) that it is a much better language for high level reasoning when coding, with a much better yield-to-character-typing ratio. Is this the reason #GenerativeAI was able to deliver with less effort? More exploration to do!!
matsuzine
in reply to Christos Argyropoulos MD, PhD • • •Christos Argyropoulos MD, PhD
in reply to matsuzine • • •If these findings, i.e. #copilot can deliver better solutions in @Perl vs. other languages, generalize, then we may see a language boost and a better job market for #perl programmers. The latter would be a result of the need to optimize the initial solutions as @mjgardner did for the #perl example over here.
Mark Gardner
in reply to Christos Argyropoulos MD, PhD • • •So the
• coached (“#PromptEngineering”)
• license-washing (training data illegally pilfered and stripped of provenance)
• stochastic parrots (#LargeLanguageModels)
need a senior #developer to help them finish the job too?
It seems like the only things “#AI” bring to the table are the shoddy #ethics of its builders and backers.
Christos Argyropoulos MD, PhD
in reply to Mark Gardner • • •Yeap, notwithstanding the shoddy ethics, I don't think it can work on its own.
At most, it will cannibalize entry level jobs (similar to what happened to human computers when digital computers came to life in the 1950s).
But assuming it takes off, the real fun will take place a few years down the road, when most of github will be composed of uncurated stochastic parrot output. I hope none of the resulting SW ends up inside a medical device
Christos Argyropoulos MD, PhD
in reply to Christos Argyropoulos MD, PhD • • •Christos Argyropoulos MD, PhD
in reply to Christos Argyropoulos MD, PhD • • •Mark Gardner
in reply to Christos Argyropoulos MD, PhD • • •Stealing others’ work isn’t “productivity.”
Then again, the culture has been infected so deeply with the “property is theft” anti-concept. So people get away with it… until the actual creators upstream go on strike.
I read a book about that once.
Perl reshared this.
matsuzine
in reply to Christos Argyropoulos MD, PhD • • •We are evaluating CoPilot at work. Only senior devs have access, because the code it generates is often wrong in subtle ways that could trip up a less experienced dev. I find that it does a great job generating comments, tests, adding log messages -- boilerplate. But again, often slightly wrong (after all, it can't think, it's just patterns).
Ethics aside, it has been at best slightly helpful, in the same way that autocompletion is helpful.
Mark Gardner
in reply to matsuzine • • •With autocomplete you’re only typing a word at a time, you check as you go, and you still decide whether to hit that “send” button.
The “#AI” dream for business software development is to remove all those human checks. The business wants to fire all their mercurial developers demanding precise requirements in favor of a black box that absolves them of guilt for remixing the code it already stole from others.
Perl reshared this.
Christos Argyropoulos MD, PhD
in reply to Mark Gardner • • •Christos Argyropoulos MD, PhD
in reply to Mark Gardner • • •Did you guys see the #FAFO (Eff Around, Find Out) when a mental health services company replaced its human councellors with #AI ? It was beautiful to behold how the disaster unfolded (in ~ 48hrs)
https://www.forbes.com/sites/chriswestfall/2023/05/31/non-profit-helpline-fires-staff-shifts-to-chatbot-solution/?sh=ed1e4d36dadc
Non-Profit Helpline Shifts To Chatbots, Then Shuts Down Rogue AI
Chris Westfall (Forbes)matsuzine
in reply to Christos Argyropoulos MD, PhD • • •Gabriel Pettier
in reply to Christos Argyropoulos MD, PhD • • •Christos Argyropoulos MD, PhD
in reply to Gabriel Pettier • • •Would be nice to have a benchmark set of problems to compare #copilot 's performance across languages.
Curtis "Ovid" Poe (he/him)
in reply to Christos Argyropoulos MD, PhD • • •