PacketFence - BTS - PacketFence
View Issue Details
IDProjectCategoryView StatusDate SubmittedLast Update
0001377PacketFenceupstreampublic2012-02-07 11:032012-02-28 14:31
Reporterobilodeau 
Assigned Toobilodeau 
PrioritynormalSeverityminorReproducibilityrandom
StatusclosedResolutionfixed 
PlatformOSOS Version
Product Version 
Target Version3.2.0Fixed in Version3.2.0 
fixed in git revision
fixed in mtn revision8219835d3324599dc23038758d2ef3324c25ae5b
Summary0001377: freeradius segfaults
Description
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xb54f4b90 (LWP 27887)]
0x00bc5726 in Perl_sv_setsv_flags () from /usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so


We're seeing this since a little while, so far we haven't got too deep into identifying root cause as we were thinking it was a freeradius problem... Now, I am getting tired of it...

Affects large environments

- Unable to reliably reproduce
- Tried freeradius 2.1.10-12 packaged and from source

Candidate solutions to move things forward:
- http://wiki.freeradius.org/Rlm_perl#Debug [^]
- recompile freeradius and perl together with a stable perl release (> 5.10)
- try to make the stacktrace better by compiling freeradius AND perl with debug/symbols enabled
- try other rlm_perl debug options as outlined on the freeradius-users list (http://www.mail-archive.com/search?q=debug+rlm_perl&l=freeradius-users%40lists.freeradius.org [^])

WORKAROUND:

Use watchdog script in addons/.
Steps To Reproduce1. Have a large wireless deployment
2. See radiusd segfault at least once a week
Additional Information
TagsNo tags attached.
Relationships
Attached Files
Issue History
Date ModifiedUsernameFieldChange
2012-02-07 11:03obilodeauNew Issue
2012-02-07 11:09obilodeauDescription Updated
2012-02-21 14:05obilodeauStatusnew => assigned
2012-02-21 14:05obilodeauAssigned To => obilodeau
2012-02-21 14:09obilodeauNote Added: 0002574
2012-02-21 14:10obilodeauDescription Updated
2012-02-21 20:54obilodeaumtn revision => 8219835d3324599dc23038758d2ef3324c25ae5b
2012-02-21 20:54obilodeauNote Added: 0002575
2012-02-21 20:54obilodeauStatusassigned => resolved
2012-02-21 20:54obilodeauFixed in Version => trunk
2012-02-21 20:54obilodeauResolutionopen => fixed
2012-02-28 14:22obilodeauTarget Version => 3.2.0
2012-02-28 14:22obilodeauFixed in Versiontrunk => 3.2.0
2012-02-28 14:31obilodeauNote Added: 0002583
2012-02-28 14:31obilodeauStatusresolved => closed

Notes
(0002574)
obilodeau   
2012-02-21 14:09   
We were able to reliably reproduce in a specific environment with a lot of load. With gdb we still tracked down the issue to rlm_perl / libperl.so. Discussing the problem to people outside of it, we got suggestions to track all native modules used by our perl module.

We found out about Sys::Syslog which uses the native syslog library by default. We decided to port our logging to the native facilities offered by radius' rlm_perl and configure radius to log to the syslog by default.
(0002575)
obilodeau   
2012-02-21 20:54   
This is tentatively marked as fixed. We think we nailed it in at least one obvious crashing case but we won't know for sure until we've deployed the fix more widely.

If you still get radiusd segfaults please let us know and re-open.
(0002583)
obilodeau   
2012-02-28 14:31   
Fixed in recently released 3.2.0.