Parallel Perl/C applications without tears using OpenMP: Controlling the OpenMP environment

https://blogs.perl.org/users/chrisarg/2024/07/parallel-perlc-applications-without-tears-using-openmp-controlling-the-openmp-environment.html

My job has led me down the rabbit hole of doing some scripting work in Perl, mainly utility tools. The challenge being that these tools need to parse several thousand source files, and doing so would take quite some time.

I initially dabbled in doing very light stuff with a perl -e one-liner from within a shell script, which meant I could use xargs. However, as my parsing needs evolved on the Perl side of things, I ended up switching to an actual Perl file, which hindered my ability to do parallel processing as our VMs did not have the Perl interpreter built with threads support. In addition, installation of any non-builtin modules such as CPAN was not possible on my target system, so I had limited possibilities, some of which I would assume to be safer and/or less quirky than this.

So then I came up with a rather ugly solution which involved invoking xargs via backticks, which then called a perl one-liner (again) for doing the more computation-heavy parts, xargs splitting the array to process into argument batches for each mini-program to process. It looked like this thus far:

my $out = `echo "$str_in" | xargs -P $num_threads -n $chunk_size perl -e ' my \@args = \@ARGV; foreach my \$arg (\@args) { for my \$idx (1 .. 100000) { my \$var = \$idx; } print "\$arg\n"; } '`; 

• No editor syntax highlighting (in my case, VSCode), since the inline program is a string.
• All variables within the inline program had to be escaped so as not to be interpolated themselves, which hindered readability quite a bit.
• Every time you would want to use this technique in different parts of the code, you'd have to copy-paste the entire shell command together with the mini-program, even if that very logic was somewhere else in your code.

After some playing around, I've come to a nifty almost-metaprogramming solution, which isn't perfect still, but fits my needs decently well:

sub processing_fct { my u/args = u/ARGV; foreach my $arg (@args) { for my $idx (1 .. 100000) { my $var = $idx; } print "A very extraordinarily long string that contains $arg words and beyond\n"; } } sub parallel_invoke { use POSIX qw{ceil}; my $src_file = $0; my $fct_name = shift; my $input_arg_array = shift; my $n_threads = shift; my $str_in = join("\n", @{$input_arg_array}); my $chunk_size = ceil(@{$input_arg_array} / $n_threads); open(my $src_fh, "<", $src_file) or die("parallel_invoke(): Unable to open source file"); my $src_content = do { local $/; <$src_fh> }; my $fct_body = ($src_content =~ /sub\s+$fct_name\s*({((?:[^}{]*(?1)?)*+)})/m)[1] or die("Unable to find function $fct_name in source file"); return `echo '$str_in' | xargs -P $n_threads -n $chunk_size perl -e '$fct_body'`; } my $out = parallel_invoke("processing_fct", \@array, $num_threads); 

My limited benchmarking has found this to be as fast if not faster than the perl-with-threads equivalent, in addition to circumventing the performance penalty for the thread safety.

I'd be curious to hear of your opinion of such method, or if you've solved a similar issue differently.

submitted by /u/Wynaan
[link] [comments]

I need some help with the old Perl Gunnar Hjalmarsson's Ringlink program on my site. The forms work, the database gets added to and everything seems ready to go except for the email functions that depend on sendmail.

I have tried several things, installed the CPAN dependencies the program needs, tried Auron SendEmail and other programs and have thoroughly confused myself.

There's a test installation on my site, with the admin and password are both 'test'. There are copies of the CGI files and probably what needs looking at are rlmain.pm, rlconfig.pm and sender.pm

I am running Apache 2.4.54 on Windows 10 with Strawberry Perl installed. I am using the last published version Ringlink (v3.4)

I know this is an old program and the project probably not worth pursuing, but I really would like to give this a go to get it working and would be grateful for any suggestions.

submitted by /u/brisray
[link] [comments]

Auron SendEmail - Auron Software Portable E-mail Freeware

The Auron SendEmail set of portable freeware tools for Windows enable you to send E-mails through from either GUI or commandline.

^{Auron Software}

my @ar_foo = ("1", "2", "3"); my $bar = "5"; my $x = $bar / scalar(@ar_foo); 

This week in PSC (150) | 2024-06-06

https://blogs.perl.org/users/psc/2024/06/this-week-in-psc-150-2024-06-06.html

Equalise an Array

https://blogs.perl.org/users/e_choroba/2024/06/equalise-an-array.html

Struggling to find what people write perl code on.

submitted by /u/SquareRaspberry3808
[link] [comments]

This is the frame body I was using:

our $frame_body = $mw->Frame(-background => $color_theme_bg, -foreground => $color_theme_fg)->pack(-side => 'top'); 

 $frame_body ->Label( -text => "@_", -font => $arial_font, -foreground => $color_theme_fg, -background => $color_theme_bg, -highlightthickness => 0, -takefocus => 0, -relief => "flat", -justify => 'center', )-> grid( -column => $mw_col_ctr, -row => $mw_row_ctr, -sticky => "nsew", ); 

Its also nice if automatically adjust incase I manually resize the window. 😀

submitted by /u/DemosaiDelacroix
[link] [comments]

Perl Weekly Challenge 272: Defang IP Address

https://blogs.perl.org/users/laurent_r/2024/06/perl-weekly-challenge-272-defang-ip-address.html

Is there a way to replace accented characters by their plain version, something like

tr/ûšḥ/ush/?

submitted by /u/Patentsmatter
[link] [comments]

Is it possible to display an image to the user, without loading all the trappings of a whole widget / event-loop environment like Prima, Tk, Wx, Win32::GUI, etc?

Specifically, I want something simple that I can execute in a BEGIN block to display a splash image to the user while the rest of the application is compiled and initializes, which takes about 5-10 seconds. The program in question is a perl Wx application running under MS Windows.

submitted by /u/sue_d_nymme
[link] [comments]

This week in PSC (149) | 2024-05-30

https://blogs.perl.org/users/psc/2024/05/this-week-in-psc-149-2024-05-30.html

Vim highlights subs with a signature params as red/orange to alert you it's incorrect. Now that Perl actually supports signatures and this is no longer a syntax error I need to find a way to disable this. This is really a Vim question, but I'm sure some other Perl nerd out there has run into this before me.

Has anyone figured out how to disable this "warning" in Vim?

submitted by /u/scottchiefbaker
[link] [comments]

Perl Weekly Challenge 271: Sort by 1 Bits

https://blogs.perl.org/users/laurent_r/2024/05/perl-weekly-challenge-271-sort-by-1-bits.html

MariaDB 10 and SQL::Translator::Producer

https://blogs.perl.org/users/usna78/2024/05/mariadb-10-and-sqltranslatorproducer.html

Perl Weekly Challenge 271: Maximum Ones

https://blogs.perl.org/users/laurent_r/2024/05/perl-weekly-challenge-271-maximum-ones.html

Perl Toolchain Summit 2024 in Lisbon

https://blogs.perl.org/users/kenichi_ishigaki/2024/05/perl-toolchain-summit-2024-in-lisbon.html

# Perl Weekly Challenge 270: Special Positions

https://blogs.perl.org/users/laurent_r/2024/05/-perl-weekly-challenge-270-special-positions.html

Hi, I recently got an offer for Senior SWE (current title at my company now) for a company that heavily utilizes Perl. I was wondering if folks from this community could offer some insight on what it's like working with Perl and also what, if any, potential long-term career implications are of becoming a Perl developer? Particularly I'm worried of pigeon-holing myself since Perl is not as heavily used in todays age and this company does not make use of modern cloud tools and deployments.

I am a Java developer (5 YOE) at a enterprise software company that is deployed in GCP. We are pretty regularly adopting new technologies so I'm gaining some valuable and relevant industry experience here but I am looking for a change and more opportunity to lead projects and mentor junior engineers.

The company seems good, great WLB, I liked the manager, and with the bonus (base is roughly the same) it would be about a ~8% TC increase plus a lot more stock (monopoly money, private RSUs).

Does anyone have experience transitioning from a Perl based company to a cloud based company with a more modern tech stack? Is this a backwards direction for me, should I continue with my Java development and instead look for opportunities that will offer more marketable skills?

Any input is appreciated, thank you for reading.

submitted by /u/Roodiestue
[link] [comments]

I've been trying to debug an issue for 3 days now, am getting nowhere, and am about to headbutt my laptop. If anyone's done any heavy lifting with Net:DBus then, for the sake of my laptop, I'd really appreciate the help!

Problem description: I have a hash table with a bunch of keys. The values relating to those keys are of different types (as in, I've cast them to dbus types). So:

my $testhash = {}; $testhash->{"xesam:albumArtist"} = [dbus_string("Tom Waits")]; $testhash->{"xesam:album"} = dbus_string("Mule Variations"); $testhash->{"xesam:trackNumber"} = dbus_int32(1); $testhash->{"xesam:artist"} = [dbus_string("Tom Waits")]; $testhash->{"xesam:title"} = dbus_string("Big in Japan"); $testhash->{"mpris:artUrl"} = dbus_string("file://mnt/storage/Music/mp3/Tom Waits/Mule Variations/folder.jpg"); $testhash->{"mpris:length"} = dbus_int64(64182857); $testhash->{"mpris:trackid"} = dbus_object_path("/0"); $testhash->{"xesam:url"} = dbus_string("file://mnt/storage/Music/mp3/Tom Waits/Mule Variations/01 - Big in Japan.mp3"); 

dbus_method("ReturnDynamicHash", [], [["dict", "string", ["variant"]]], $IFACE); sub ReturnDynamicHash { my $self = shift; print "Object: ReturnDynamicHash called.\n"; my $return = {}; my @keys = keys(%{$testhash}); my $count = scalar(@keys); if ($count) { foreach my $key (@keys) { $return->{$key} = $testhash->{$key}; } } return $return; } 

% dbus-send ....... .ReturnDynamicHash array [ dict entry( xesam:trackNumber variant int32 1 ) dict entry( mpris:trackid variant /0 ) dict entry( xesam:albumArtist variant array [ Tom Waits ] ) dict entry( xesam:album variant Mule Variations ) dict entry( mpris:length variant int64 64182857 ) dict entry( xesam:url variant file://mnt/storage/Music/mp3/Tom Waits/Mule Variations/01 - Big in Japan.mp3 ) dict entry( mpris:artUrl variant file://mnt/storage/Music/mp3/Tom Waits/Mule Variations/folder.jpg ) dict entry( xesam:artist variant array [ Tom Waits ] ) dict entry( xesam:title variant Big in Japan ) ] 

dbus_property("StaticHashProperty", [["dict", "string", ["variant"]]], "read", $IFACE); sub StaticHashProperty { print "Object: StaticHashProperty accessed.\n"; my $return = {}; my @keys = keys(%{$testhash}); my $count = scalar(@keys); if ($count) { foreach my $key (@keys) { $return->{$key} = $testhash->{$key}; } } return $return; } 

From the dbus-send client I get

Error org.freedesktop.DBus.Error.NoReply: Remote peer disconnected 

dbus[93409]: Array or variant type requires that type array be written, but object_path was written. The overall signature expected here was 'a{sas}' and we are on byte 4 of that signature. D-Bus not built with -rdynamic so unable to print a backtrace Aborted (core dumped) 

Since the Method works I know Net::DBus can handle these sorts of return values, but for some reason, as a property, it just isn't working. I also know that other applications do implement this interface, including this Property, successfully, so I know this isn't a limitation of DBus.

I've been looking at the code in Net::DBus that handles serialization, assuming there must be some difference between how Properties and Methods are handled, but can't see anything obvious.

Anyone? Any idea? Literally anything at all? Thank you!!!!!

submitted by /u/Flashy_Boot
[link] [comments]

The following is a quick ramble before I get into client work, but might give you an idea of how AI is being used today in companies. If you have an questions about Generative AI, let me know!

The work to make the OpenAI API (built on Nelson Ferraz's OpenAPI::Client::OpenAI module) is going well. I now have working example of transcribing audio using OpenAI's whisper-1 model, thanks to the help of Rabbi Veesh.

Using a 7.7M file which is about 16 minutes long, the API call takes about 45 seconds to run and costs $0.10 USD to transcribe. The resulting output has 2,702 words and seems accurate.

Next step is using an "instruct" model to appropriately summarize the results ("appropriate" varies wildly across use cases). Fortunately, we already have working examples of this. Instruct models tend to be more correct in their output than chat models, assuming you have a well-written prompt. Anecdotally, they may have smaller context windows because they're not about remembering a long conversation, but I can't prove that.

Think about the ROI on this. The transcription and final output will cost about 11 cents and take a couple of minutes. You'll still need someone to review it. However, think of the relatively thankless task of taking meeting minutes and producing a BLUF email for the company. Hours of expensive human time become minutes of cheap AI time. Multiply this one task by the number of times per year you have to do it. Further, consider how many other "simple tasks" can be augmented via AI and you'll see why it's becoming so powerful. A number of studies show that removing many of these simple tasks from people's plates, allowing them to focus on the "big picture," is resulting in greater morale and productivity.

When building AI apps, OpenAPI::Client::OpenAI should be thought of as a "low-level" module, similar to DBIx::Class. It should not be used directly in your code, but hidden behind an abstraction layer. Do not use it directly.

I tell my clients that their initial work with AI should be a tactical "top-down mandate/bottom-up implementation." This gives them the ability to start learning how AI can be used in different parts of their organization, given that marketing, HR, IT, and other departments all have different needs.

Part of this tactical approach is learning how to build AI data pipelines. With OpenAI publishing their OpenAPI spec, and with Perl using that, we can bring much of the power of enterprise-level AI needs to companies using Perl. It's been far too long that Perl has languished in the AI space.

Next, I need to investigate doing this with Gemini and/or Claude, but not now.

Note, if you're not familiar with the BLUF format, it's a style of writing email that is well-suited for email in a company that is sent to many people. It's "bottom-line up front" so that people can see the main point and decide if the rest of the email is relevant to them. It makes for very effiicient email communication.

submitted by /u/OvidPerl
[link] [comments]

BLUF: THE method for better and effective written communication

Communication in remote work is all about written communication. That’s the big shift that happened regarding how we communicate. The Economist and Information Week are two examples of many…

^{Gabriela De Luca (Medium)}

Hi all,

Nelson Ferraz has been working with generative AI for a while. I've started collaborating with him on his OpenAI modules. He wrote a module named OpenAI::API, but it required manually writing the code for all of the behavior. With the size of the OpenAI API, the rapid evolution, of said API, the birth of new models and the deprecation of old models, this approach turned out to be unmaintainable.

Thus, that module was deprecated in favor of Nelson's OpenAPI::Client::OpenAI module. Throw the 13K+ lines OpenAPI spec for OpenAI at it and it just works. Further, the module is pretty much a single Perl class rather than a bunch of hand-crafted code.

CPAN authors know it can be hard to keep modules up-to-date (mea culpa, mea culpa!) and this module is no exception. I need this module so I offered to collaborate and created a PR to update it to version 2.0.0 of the OpenAI spec. It now passes all the tests (for those wondering, you need an OpenAI key and it costs $0.04 USD to run the test suite).

In trying to build a Whisper pipeline for that, I found that I couldn't. There was a PR for Whisper support for the older module, but for the newer one, I can't figure out how to get it to issue a request with multipart/form-data support. I've noted the issue in the PR.

If anyone would like to see OpenAI support for Perl, we would dearly love to collaborate with you to make this happen.

submitted by /u/OvidPerl
[link] [comments]

Nelson Ferraz

CPAN Author

^metacpan.org

MariaDB 10 and SQL::Translator::Parser

https://blogs.perl.org/users/usna78/2024/05/mariadb-10-and-sqltranslatorparser.html

This week in PSC (148) | 2024-05-16

https://blogs.perl.org/users/psc/2024/05/this-week-in-psc-148-2024-05-16.html

Confirming The LPW 2024 Venue & Date

https://blogs.perl.org/users/lpw/2024/05/confirming-the-lpw-2024-venue-date.html

Perl Weekly Challenge 269: Distribute Elements

https://blogs.perl.org/users/laurent_r/2024/05/perl-weekly-challenge-269-distribute-elements.html

Perl Weekly Challenge 269: Bitwise OR

https://blogs.perl.org/users/laurent_r/2024/05/perl-weekly-challenge-269-bitwise-or.html

I want to repeat a process for every key in a hash, with numeric keys. So there are 3 possibilities, with 3 if, and each one compares the value of the index of an array, so that if that position eq to "sp", "sp2" or "sp3" it will search in a document some value so then it can be printed. It doesn´t work and every times gives me only one value, i would like to get the values that correspond with the hash. For example the hash could be %grupos=(1,'A',2,'G',3,'J')

and the array @hibridaciones=("sp","sp2",sp3")

The document .txt (simplified) is:

HS0.32 CS0,77 CD0.62 CT0,59 C10,77 C20,62 C30,59 OS0.73 OD0,6 O10,73 O20,6 NS0.75

The code is:

open (covalencia,"<", "cov.txt") or die "$!\n"; print keys %grupos; keys %grupos; foreach my $z (keys %grupos) { print "\n$z\n"; if (@hibridaciones[my $z-1] eq “sp") { while (my $line = <covalencia>) { if ( $line=~/C1/) { $line =~s/C1//; $radio=$line; print "\n$radio"; } } } if (@hibridaciones[my $z-1] eq "sp2") { while (my $line = <covalencia>) { if ($line=~/C2/) { $line =~s/C2//; $radio=$line; print "\n$radio"; } } } if (@hibridaciones[my $z-1] eq "sp3") { while (my $line = <covalencia>) { if ($line=~/C3/) { $line =~s/C3//; $radio=$line; print "\n$radio"; } } } } close (covalencia);

submitted by /u/SamuchRacoon
[link] [comments]

Making a Super Cal if Rage Will Stick Ex Paella Down Us

https://blogs.perl.org/users/saif/2024/05/making-a-super-cal-if-rage-will-stick-ex-paella-down-us.html

On this site: https://codegolf.stackexchange.com/questions/246845/convert-json-object-of-directories-to-list-of-paths

There is an interesting perl solution but I'm trying to wrap my head around it and am baffled.

map/}/?/{/&&say($s)..$s=~s|[^/]+/$||:($s.=s/"//gr."/"),/{?}|".+?"/g When I do a simple test: my $s = <some json string> .. the map above I don't get any output. Thanks in advance 

Convert JSON object of directories to list of paths

Task The input consists of a JSON object, where every value is an object (eventually empty), representing a directory structure. The output must be a list of the corresponding root-to-leaf paths.

^{Code Golf Stack Exchange}

I have a bunch of files that I am parsing out a simple tag
<institution content-type="division">(.+?)?<\/institution>

Everything is going fine until I hit this result:
2) School of Chemical & Biomolecular Engineering

This is complaining because it thinks the ) is unmatched. How do I get it to accept whatever is in the capture? I tried playing around with \Q and \E but that isn't working. For clarity, I won't know what is in this tag, it could be anything at all and the tag is simple enough that I don't want to use some XML parser.

submitted by /u/sirhalos
[link] [comments]

This Week in PSC (147)

https://blogs.perl.org/users/psc/2024/05/this-week-in-psc-147.html

I'm experimenting with different frameworks and I've come across this wonderful catalyst framework based on Perl, does anyone know of a front end (which is website, forums, etc.) coupled with a back end (tracker engine) for use with modern Perl? I have a PHP based one and was wondering if it was possible to do the same using Perl.

submitted by /u/Otherwise-Tune-257
[link] [comments]

MariaDB 10 and Perl DBIx::Class::Schema::Loader

https://blogs.perl.org/users/usna78/2024/05/mariadb-10-and-perl-dbixclassschemaloader.html

Perl Weekly Challenge 268: Number Game

https://blogs.perl.org/users/laurent_r/2024/05/perl-weekly-challenge-268-number-game.html