Skip to main content




My job has led me down the rabbit hole of doing some scripting work in Perl, mainly utility tools. The challenge being that these tools need to parse several thousand source files, and doing so would take quite some time.

I initially dabbled in doing very light stuff with a perl -e one-liner from within a shell script, which meant I could use xargs. However, as my parsing needs evolved on the Perl side of things, I ended up switching to an actual Perl file, which hindered my ability to do parallel processing as our VMs did not have the Perl interpreter built with threads support. In addition, installation of any non-builtin modules such as CPAN was not possible on my target system, so I had limited possibilities, some of which I would assume to be safer and/or less quirky than this.

So then I came up with a rather ugly solution which involved invoking xargs via backticks, which then called a perl one-liner (again) for doing the more computation-heavy parts, xargs splitting the array to process into argument batches for each mini-program to process. It looked like this thus far:

my $out = `echo "$str_in" | xargs -P $num_threads -n $chunk_size perl -e ' my \@args = \@ARGV; foreach my \$arg (\@args) { for my \$idx (1 .. 100000) { my \$var = \$idx; } print "\$arg\n"; } '`; 

However, this had some drawbacks:
  • No editor syntax highlighting (in my case, VSCode), since the inline program is a string.
  • All variables within the inline program had to be escaped so as not to be interpolated themselves, which hindered readability quite a bit.
  • Every time you would want to use this technique in different parts of the code, you'd have to copy-paste the entire shell command together with the mini-program, even if that very logic was somewhere else in your code.

After some playing around, I've come to a nifty almost-metaprogramming solution, which isn't perfect still, but fits my needs decently well:

sub processing_fct { my u/args = u/ARGV; foreach my $arg (@args) { for my $idx (1 .. 100000) { my $var = $idx; } print "A very extraordinarily long string that contains $arg words and beyond\n"; } } sub parallel_invoke { use POSIX qw{ceil}; my $src_file = $0; my $fct_name = shift; my $input_arg_array = shift; my $n_threads = shift; my $str_in = join("\n", @{$input_arg_array}); my $chunk_size = ceil(@{$input_arg_array} / $n_threads); open(my $src_fh, "<", $src_file) or die("parallel_invoke(): Unable to open source file"); my $src_content = do { local $/; <$src_fh> }; my $fct_body = ($src_content =~ /sub\s+$fct_name\s*({((?:[^}{]*(?1)?)*+)})/m)[1] or die("Unable to find function $fct_name in source file"); return `echo '$str_in' | xargs -P $n_threads -n $chunk_size perl -e '$fct_body'`; } my $out = parallel_invoke("processing_fct", \@array, $num_threads); 

All parallel_invoke() does is open it's own source file, finds the subroutine declaration, and then passes the function body captured by the regex (which isn't too pretty, but it was necessary to reliably match a balanced construct of nested brackets) - to the xargs perl call.

My limited benchmarking has found this to be as fast if not faster than the perl-with-threads equivalent, in addition to circumventing the performance penalty for the thread safety.

I'd be curious to hear of your opinion of such method, or if you've solved a similar issue differently.

submitted by /u/Wynaan
[link] [comments]



List of new CPAN distributions – Jun 2024 submitted by /u/perlancar
[link] [comments]


I need some help with the old Perl Gunnar Hjalmarsson's Ringlink program on my site. The forms work, the database gets added to and everything seems ready to go except for the email functions that depend on sendmail.

I have tried several things, installed the CPAN dependencies the program needs, tried Auron SendEmail and other programs and have thoroughly confused myself.

There's a test installation on my site, with the admin and password are both 'test'. There are copies of the CGI files and probably what needs looking at are rlmain.pm, rlconfig.pm and sender.pm

I am running Apache 2.4.54 on Windows 10 with Strawberry Perl installed. I am using the last published version Ringlink (v3.4)

I know this is an old program and the project probably not worth pursuing, but I really would like to give this a go to get it working and would be grateful for any suggestions.

submitted by /u/brisray
[link] [comments]



submitted by /u/niceperl
[link] [comments]


A quick demo I threw togetherI threw together a quick proof of concept for myself writing out a very simple Entity Component System (ECS) and implementing the flocking simulation on top of it. I liked how it came together so well I wrote some prose around it and decided to share.

Note: this is using features from the soon-to-be-released 5.40.0 (RC1 dropped last Friday).

submitted by /u/perigrin
[link] [comments]



submitted by /u/niceperl
[link] [comments]


Hi, I recently got an offer for Senior SWE (current title at my company now) for a company that heavily utilizes Perl. I was wondering if folks from this community could offer some insight on what it's like working with Perl and also what, if any, potential long-term career implications are of becoming a Perl developer? Particularly I'm worried of pigeon-holing myself since Perl is not as heavily used in todays age and this company does not make use of modern cloud tools and deployments.

I am a Java developer (5 YOE) at a enterprise software company that is deployed in GCP. We are pretty regularly adopting new technologies so I'm gaining some valuable and relevant industry experience here but I am looking for a change and more opportunity to lead projects and mentor junior engineers.

The company seems good, great WLB, I liked the manager, and with the bonus (base is roughly the same) it would be about a ~8% TC increase plus a lot more stock (monopoly money, private RSUs).

Does anyone have experience transitioning from a Perl based company to a cloud based company with a more modern tech stack? Is this a backwards direction for me, should I continue with my Java development and instead look for opportunities that will offer more marketable skills?

Any input is appreciated, thank you for reading.

submitted by /u/Roodiestue
[link] [comments]




I've been trying to debug an issue for 3 days now, am getting nowhere, and am about to headbutt my laptop. If anyone's done any heavy lifting with Net:DBus then, for the sake of my laptop, I'd really appreciate the help!

Problem description: I have a hash table with a bunch of keys. The values relating to those keys are of different types (as in, I've cast them to dbus types). So:

my $testhash = {}; $testhash->{"xesam:albumArtist"} = [dbus_string("Tom Waits")]; $testhash->{"xesam:album"} = dbus_string("Mule Variations"); $testhash->{"xesam:trackNumber"} = dbus_int32(1); $testhash->{"xesam:artist"} = [dbus_string("Tom Waits")]; $testhash->{"xesam:title"} = dbus_string("Big in Japan"); $testhash->{"mpris:artUrl"} = dbus_string("file://mnt/storage/Music/mp3/Tom Waits/Mule Variations/folder.jpg"); $testhash->{"mpris:length"} = dbus_int64(64182857); $testhash->{"mpris:trackid"} = dbus_object_path("/0"); $testhash->{"xesam:url"} = dbus_string("file://mnt/storage/Music/mp3/Tom Waits/Mule Variations/01 - Big in Japan.mp3"); 

I've created a DBus service, and have successfully implemented a method that returns that hash table ($IFACE is the interface name I'm using for all of my test methods):
dbus_method("ReturnDynamicHash", [], [["dict", "string", ["variant"]]], $IFACE); sub ReturnDynamicHash { my $self = shift; print "Object: ReturnDynamicHash called.\n"; my $return = {}; my @keys = keys(%{$testhash}); my $count = scalar(@keys); if ($count) { foreach my $key (@keys) { $return->{$key} = $testhash->{$key}; } } return $return; } 

As a DBus method, this works perfectly:
% dbus-send ....... .ReturnDynamicHash array [ dict entry( xesam:trackNumber variant int32 1 ) dict entry( mpris:trackid variant /0 ) dict entry( xesam:albumArtist variant array [ Tom Waits ] ) dict entry( xesam:album variant Mule Variations ) dict entry( mpris:length variant int64 64182857 ) dict entry( xesam:url variant file://mnt/storage/Music/mp3/Tom Waits/Mule Variations/01 - Big in Japan.mp3 ) dict entry( mpris:artUrl variant file://mnt/storage/Music/mp3/Tom Waits/Mule Variations/folder.jpg ) dict entry( xesam:artist variant array [ Tom Waits ] ) dict entry( xesam:title variant Big in Japan ) ] 

However, the interface I'm implementing requires that a DBus Property return that hashtable, not a method:
dbus_property("StaticHashProperty", [["dict", "string", ["variant"]]], "read", $IFACE); sub StaticHashProperty { print "Object: StaticHashProperty accessed.\n"; my $return = {}; my @keys = keys(%{$testhash}); my $count = scalar(@keys); if ($count) { foreach my $key (@keys) { $return->{$key} = $testhash->{$key}; } } return $return; } 

and this doesn't work.

From the dbus-send client I get

Error org.freedesktop.DBus.Error.NoReply: Remote peer disconnected 

and from the Perl server stderr i get:
dbus[93409]: Array or variant type requires that type array be written, but object_path was written. The overall signature expected here was 'a{sas}' and we are on byte 4 of that signature. D-Bus not built with -rdynamic so unable to print a backtrace Aborted (core dumped) 

Now, this error is coming from libdbus itself, not the Perl wrapper (though could of course still be a bug in the Perl module that's causing the error). It seems to have entirely the wrong signature ( a{sas}, not a{sv} as defined above the Property method) and therefore appears to be complaining that the type of one of the values is wrong (each time I run it I get a slightly different error; I think it's deducing the signature from the first key-value pair it pulls from the hash and assumes they should all be the same - so if the first pair it pulls has a uint64 value, then it complains that the next pair doesn't also have a uint64 value).

Since the Method works I know Net::DBus can handle these sorts of return values, but for some reason, as a property, it just isn't working. I also know that other applications do implement this interface, including this Property, successfully, so I know this isn't a limitation of DBus.

I've been looking at the code in Net::DBus that handles serialization, assuming there must be some difference between how Properties and Methods are handled, but can't see anything obvious.

Anyone? Any idea? Literally anything at all? Thank you!!!!!

submitted by /u/Flashy_Boot
[link] [comments]



Mo utilities for email.

Changes for 0.02 - 2024-04-26T23:02:53+02:00

  • Add tests for error parameters.
  • Rewrite the tests so that the functional tests are first and then the errors.







Hi! Asking for a wisdom here...

We have a module that modifies signal handler $SIG{__DIE__} to log information and to die afterwards. Hundreds of scripts relied on this module which worked fine in perl 5.10.1.

Recently we had the opportunity to install several Perl versions but unfortunately a large number of scripts that used to work with Perl 5.10.1 now behave differently:

  • Failed in 5.14.4: /home/dev/perl-5.14.4/bin/perl -wc test.pl RECEIVED SIGNAL - S_IFFIFO is not a valid Fcntl macro at /home/dev/perl-5.14.4/lib/5.14.4/File/stat.pm line 41
  • Worked without changes in 5.26.3: /home/dev/perl-5.26.3/bin/perl -wc test.pl test.pl syntax OK
  • Worked without changes in 5.38.2: /home/dev/perl-5.38.2/bin/perl -wc test.pl test.pl syntax OK

Many of the scripts can only be updated to 5.14.4 due to the huge jumps between 5.10 and 3.58; But we are stuck on that failures.

Was there an internal Perl change in 5.14 which cause the failures but works on other recent versions without any update on the scripts?

Cheerio!

submitted by /u/Longjumping_Army_525
[link] [comments]



Sanity-check calling context

Changes for 0.04

  • (no code changes)
  • Switched to MIT license.
  • Switched README from POD to Markdown.
  • Removed Travis CI.



Sort lines of text by a Comparer module

Changes for 0.002 - 2024-03-07

  • No functional changes.
  • [doc] Mention some related links.


An assortment of date-/time-related CLI utilities

Changes for 0.128 - 2024-03-07

  • [clis strftime, strftimeq] Use localtime() instead of gmtime(). We can still show UTC using "TZ=UTC strftime ...".



I understand that many disagree with this statement, but it really makes it easier to build distributions for people who not monks. Wish the documentation was more detailed

submitted by /u/ReplacementSlight413
[link] [comments]



Sah schemas related to BCA (Bank Central Asia) bank

Changes for 0.002 - 2024-04-03

  • Rename module/dist Sah-Schema{s,Bundle}-* following rename of Sah-Schema{s,Bundle} (for visual clarity and consistency with naming of other bundles).



search nested hashref/arrayref structures using JSONPath

Changes for 1.0.5 - 2024-04-22T16:10:46-05:00



simulating paper and pencil techniques for basic arithmetic operations

Changes for 0.01 - 2024-04

  • First version, with the four basic operations, plus square-root, GCD and radix conversion. And HTML rendering


Use a type to validate values in a deep comparison.

Changes for 1.0.1 - 2024-04-22

  • Add Test2::Tools::Type





An open source web-based network management tool.

Changes for 2.076000 - 2024-04-22

  • NEW FEATURES
  • ENHANCEMENTS
  • BUG FIXES


Silverpeak Orchestrator REST API client library

Changes for 0.011000 - 2024-04-22T17:58:39+02:00

  • add support for version 9.3+ API endpoints


A module that performs semantic similarity in PXF/BFF data structures and beyond (JSON|YAML)

Changes for 0.07 - 2024-04-22T00:00:00Z

  • Excluded keys with {} or [] values (e.g. subject.vitalStatus: {})
  • Precompiled regex patterns where possible
  • Schema validation errors related to weights are now directed to STDERR
  • Added support for variables with non-word characters (e.g., "Survey.Timestamp.1")



Pango style markup formatting

Changes for 0.036 - 2024-04-22

  • Introducing struts. Update SYNOPOSIS example to modern PDF::API2 conventions. Fix/enhance drawing of brackgrounds. Fix problem with wrongly calculated underlining.


My environment is perl/5.18.2 on CentOs 7

I'm trying to use a SWIG generated module in perl, which has a c plus plus backend. The backend.cpp sets an environment variable, $MY_ENV_VAR =1

But when I try to access this in perl, using $ENV{MY_ENV_VAR} this is undef.

However doing something like print echo $MY_ENV_VAR works

So the variable is set in the process, but it's not reflected automatically since nothing updates the $ENV data structure.

I'm assuming it may work using some getEnv like mechanism, but is there a way to reset/ refresh the $ENV that it rebuilds itself from the current environment?

submitted by /u/sarcasmwala
[link] [comments]



By default cpanm drops Perl modules into ~/perl5. How do I tell cpanm to use a different location, such as ~/.local/share/perl5 instead?

submitted by /u/s-ro_mojosa
[link] [comments]



Get stock and mutual fund quotes from various exchanges

Changes for 1.61_01 - 2024-04-21T17:50:16-07:00

  • YahooWeb.pm - Issue #377. Modified YahooWeb to account for changes from Yahoo.
#377


Sah schemas related to SortKey

Changes for 0.002 - 2024-03-07

  • Rename module/dist Sah-Schema{s,Bundle}-SortKey.


Perl Data Language

Changes for 2.088 - 2024-04-22

  • Slatec::ch{ic,sp} work arrays now [t]
  • add Slatec::bvalu
  • add Func::{pchip,spline}, and a demo
  • add Ufunc::diff2
  • extra ) on end of Pars now an error
  • PP dim sizes can be =CALC(...) instead of explicit RedoDimsCode
  • PP add loop(n=start:end:inc) idiom to stop not at end and have non-1 inc
  • updated README.md - thanks @falsifian
  • support T_PTROBJ in typemap


an interactive plotting widget and library for PDL and Prima

Changes for 0.21 - 2024-04-22

  • demo now only loads Prima when run, reducing perldl problems


demonstrate PDL::Graphics::PLplot capabilities

Changes for 0.84 - 2024-04-22

  • add perldl demo
  • fix plFreeGrid to use free that matches the allocator
  • pl{Alloc,Free}{,2d}Grid error if not match allocating and freeing routine
  • minimum PDL 2.083 due to output OtherPars