X Tutup
Skip to content

Donate cpu server crash locations#2276

Merged
versat merged 8 commits intodanmar:masterfrom
rikardfalkeborn:donate-cpu-server-crash-locations
Oct 18, 2019
Merged

Donate cpu server crash locations#2276
versat merged 8 commits intodanmar:masterfrom
rikardfalkeborn:donate-cpu-server-crash-locations

Conversation

@rikardfalkeborn
Copy link
Copy Markdown
Contributor

Refactor donate-cpu-server.py slightly, then add the number of crashes per file and line.

I think this could be useful when there are lots of crashes in daca@home (like now for example), to get an overview where the crashes occur, if there are multiple problems, etc. Some input on this would be good (for example, should it be formatted differently? Would it be useful to generate new html-pages for each file/line with links to the packages that crashed?). The changes are easiest to review if you ignore white-space changes, since a fairly large chunk had its indentation changed in crashReport().

Example output:

Crash report

Package                                  Date        Time   1.89   Head 
ace                                      2019-10-16 19:43         Crash 
actor-framework                          2019-10-16 19:58         Crash 
amarok                                   2019-10-16 21:42         Crash 
libffado                                 2019-10-16 21:44         Crash 
zathura                                  2019-10-16 22:25         Crash 
HEAD Crash locations                     Count      
lib/symboldatabase.h:342                 2          
build/valueflow.cpp:3775                 1          
lib/token.h:201                          1          

This adds the number of crashes per file and line to the crash report.

Example output:

	HEAD Crash location                      Count
	lib/symboldatabase.h:342                 2
	build/valueflow.cpp:3775                 1
	lib/token.h:201                          1
@danmar
Copy link
Copy Markdown
Owner

danmar commented Oct 17, 2019

sounds good!

@amai2012
Copy link
Copy Markdown
Collaborator

What about creating a hash from the callstack? It could be shown in the crash report to quickly identify problems.

@rikardfalkeborn
Copy link
Copy Markdown
Contributor Author

What about creating a hash from the callstack? It could be shown in the crash report to quickly identify problems.

Yes, that would make it possible to separate crashes in say Token() depending on which checker called it. But I'd have to filter out args and addresses from the call stack, right? Perhaps we could print the call stacks below to make it easy to find?

Or what about just printing the entire call stack and the number of crashes that resulted in it? With a cap of the depth to avoid printing out everything when we have infinite recursion...

Something like:

10 crashes
Call stack here

3 crashes
Another call stack here

Etc.

@firewave
Copy link
Copy Markdown
Collaborator

Yes, that would make it possible to separate crashes in say Token() depending on which checker called it. But I'd have to filter out args and addresses from the call stack, right? Perhaps we could print the call stacks below to make it easy to find?

Or what about just printing the entire call stack and the number of crashes that resulted in it? With a cap of the depth to avoid printing out everything when we have infinite recursion...

Something like:

10 crashes
Call stack here

3 crashes
Another call stack here

Etc.

I was also thinking about doing a report like this. I also wasn't sure how to handle it, but my idea was to make a different report which just lists the library with the date and the signature of the function the crash happened in so you get a brief overview.

@versat
Copy link
Copy Markdown
Collaborator

versat commented Oct 17, 2019

Here is a crash report created with Cygwin:

2019-10-17 12:31
ftp://ftp.se.debian.org/debian/pool/main/a/actor-framework/actor-framework_0.16.3.orig.tar.gz
cppcheck-options: -j1 --library=posix --library=gnu --library=libcurl --library=qt -D__GNUC__ --showtime=top5 --check-library --inconclusive --enable=style,information --platform=unix64 --template=daca2 -rp=temp temp
platform: CYGWIN_NT-6.1-7601-3.0.7-338.x86_64-x86_64-64bit-WindowsPE
python: 3.6.9
client-version: 1.1.38
cppcheck: head 1.89
head-info: d1fce6b7a (2019-10-17 07:08:10 +0200)
count: Crash! 238
elapsed-time: -11.0 3.9
head-timing-info:

old-timing-info:
 Tokenizer::createTokens: 1.485s (avg. 0.00462617s - 321 result(s))
 Tokenizer::simplifyTokens1: 1.009s (avg. 0.00317296s - 318 result(s))
 Tokenizer::simplifyTokens1::ValueFlow: 0.311s (avg. 0.00128512s - 242 result(s))
 Check function usage::runChecks: 0.251s (avg. 0.00103719s - 242 result(s))
 Tokenizer::simplifyTokens1::createSymbolDatabase: 0.139s (avg. 0.00057438s - 242 result(s))
 Overall time: 3.882s

head results:
Checking temp/actor-framework-0.16.3/libcaf_io/src/basp_broker.cpp: __GNUC__=1...

Thread 1 "cppcheck" received signal SIGSEGV, Segmentation fault.
valueFlowReverse (tokenlist=tokenlist@entry=0xffffb440, tok=tok@entry=0x6007cf8b0, varToken=varToken@entry=0x6007cfb10, val=..., val2=..., errorLogger=errorLogger@entry=0xffffbc50, settings=settings@entry=0xffffbe48) at build/valueflow.cpp:3775
3775	                if (!assignTok->hasKnownValue()) {
#0  valueFlowReverse (tokenlist=tokenlist@entry=0xffffb440, tok=tok@entry=0x6007cf8b0, varToken=varToken@entry=0x6007cfb10, val=..., val2=..., errorLogger=errorLogger@entry=0xffffbc50, settings=settings@entry=0xffffbe48) at build/valueflow.cpp:3775
#1  0x00000001005f2b8a in valueFlowBeforeCondition (tokenlist=tokenlist@entry=0xffffb440, symboldatabase=symboldatabase@entry=0x600801fe0, errorLogger=errorLogger@entry=0xffffbc50, settings=settings@entry=0xffffbe48) at build/valueflow.cpp:4092
#2  0x00000001005f6dc7 in ValueFlow::setValues (tokenlist=tokenlist@entry=0xffffb440, symboldatabase=0x600801fe0, errorLogger=0xffffbc50, settings=0xffffbe48) at build/valueflow.cpp:8403
#3  0x00000001005bf63e in Tokenizer::simplifyTokens1 (this=this@entry=0xffffb440, configuration=...) at build/tokenize.cpp:11095
#4  0x00000001004d92ae in CppCheck::checkFile (this=this@entry=0xffffbc50, filename=..., cfgname=..., fileStream=...) at build/cppcheck.cpp:513
#5  0x00000001004dcd29 in CppCheck::check (this=this@entry=0xffffbc50, path=...) at build/cppcheck.cpp:197
#6  0x0000000100600af8 in CppCheckExecutor::check_internal (this=this@entry=0xffffc800, cppcheck=..., argv=argv@entry=0xffffcad0) at cli/cppcheckexecutor.cpp:884
#7  0x0000000100601842 in CppCheckExecutor::check (this=this@entry=0xffffc800, argc=argc@entry=16, argv=argv@entry=0xffffcad0) at cli/cppcheckexecutor.cpp:198
#8  0x0000000100671c1e in main (argc=16, argv=0xffffcad0) at cli/main.cpp:95

DONE

@rikardfalkeborn
Copy link
Copy Markdown
Contributor Author

rikardfalkeborn commented Oct 17, 2019

I decided to try to print the entire call stack.

I've tested by downloading all available crash files on the daca@home server, oneliner below

for f in $(curl -s http://cppcheck1.osuosl.org:8000/crash.html | grep -o '".*"' | tr -d '"'); do curl -s -O http://cppcheck1.osuosl.org:8000/$f; done

Now, after the list of all crashes, stack traces are printed too, with the most common one at the top together with the packages that had that specific stack trace.

Example output (the packages are linked in the report, but I couldn't be bothered trying to get that to work in this comment). As you can see, the line with the packages is quite long for the most common crash. Also, it prints Packages even if the stack trace comes from a single package :

Stack traces

Packages: insighttoolkit intel-mediasdk isc-kea juce kradio4 kraken2 kvirc libassa libc++ libffado libgff liblivemedia libopencsd libshevek libtgvoip libtorrent-rasterbar libunivalue libvma lilypond llvm-toolchain-3.4 llvm-toolchain-3.5 llvm-toolchain-3.7 llvm-toolchain-3.8 llvm-toolchain-3.9 llvm-toolchain-4.0 llvm-toolchain-5.0 llvm-toolchain-6.0 llvm-toolchain-7 llvm-toolchain-8 llvm-toolchain-9 llvm-toolchain-snapshot log4cxx mailavenger mariadb-10.1 mariadb-10.3 matplotlib matplotlib2 maxflow mednafen minidjvu mira mixxx mongodb monkeystudio motif mpqc3 mysql-5.5 nautic ncbi-blast+ ncbi-tools6 netgen nethack newsbeuter ns3 nss ntl nx-libs objcryst-fox oce oclgrind octave opencascade opencv openms openscenegraph openscenegraph-3.4 openzwave ossim otb pari pcl pekwm polygraph postbooks pythia8 python-enable qpxtool r-cran-stringi resiprocate restbed rheolef robojournal rocksdb root-system
0x00005555557e8cfd in valueFlowForwardVariable (startToken=startToken@entry=0x5555564f8890, endToken=endToken@entry=0x5555585c8300, var=0x0, varid=70, values=std::__cxx11::list = {...}, constValue=false, subFunction=false, tokenlist=, errorLogger=, settings=) at build/valueflow.cpp:4363
4363	        else if ((var->isGlobal() || tok2->str() == MatchCompiler::makeConstString("asm")) && match77(tok2) && match87(tok2->linkAt(1))) {
#0 valueFlowForwardVariable (...) at build/valueflow.cpp:4363
#1 valueFlowForward (...) at build/valueflow.cpp:5096
#2 valueFlowReverse (...) at build/valueflow.cpp:3784
#3 valueFlowBeforeCondition (...) at build/valueflow.cpp:4092
#4 ValueFlow::setValues (...) at build/valueflow.cpp:8403
#5 Tokenizer::simplifyTokens1 (...) at build/tokenize.cpp:11095
#6 CppCheck::checkFile (...) at build/cppcheck.cpp:513
#7 CppCheck::check (...) at build/cppcheck.cpp:197
#8 CppCheckExecutor::check_internal (...) at cli/cppcheckexecutor.cpp:884
#9 CppCheckExecutor::check (...) at cli/cppcheckexecutor.cpp:198
#10 main (...) at cli/main.cpp:95

Packages: amarok android-platform-art android-platform-frameworks-native bind bind9 boxbackup qtox quassel rdkit ring rosegarden rtags soprano spoa syncevolution wesnoth-1.14
Scope::getVariable (this=0x0, varname="data") at build/symboldatabase.cpp:5736
5736	    for (iter = varlist.begin(); iter != varlist.end(); ++iter) {
#0 Scope::getVariable (...) at build/symboldatabase.cpp:5736
#1 SymbolDatabase::createSymbolDatabaseSetVariablePointers (...) at build/symboldatabase.cpp:3114
#2 SymbolDatabase::SymbolDatabase (...) at build/symboldatabase.cpp:1997
#3 Tokenizer::createSymbolDatabase (...) at build/tokenize.cpp:19512
#4 Tokenizer::simplifyTokens1 (...) at build/tokenize.cpp:11081
#5 CppCheck::checkFile (...) at build/cppcheck.cpp:513
#6 CppCheck::check (...) at build/cppcheck.cpp:197
#7 CppCheckExecutor::check_internal (...) at cli/cppcheckexecutor.cpp:884
#8 CppCheckExecutor::check (...) at cli/cppcheckexecutor.cpp:198
#9 main (...) at cli/main.cpp:95

Example output:

	Packages: psi xenomai
	Token::hasKnownValue (this=0x0) at lib/token.h:988
	988	        return mImpl->mValues && std::any_of(mImpl->mValues->begin(), mImpl->mValues->end(), std::mem_fn(&ValueFlow::Value::isKnown));
	#0 Token::hasKnownValue (...) at lib/token.h:988
	#1 valueFlowReverse (...) at build/valueflow.cpp:3775
	danmar#2 valueFlowBeforeCondition (...) at /usr/include/c++/8/bits/stl_list.h:301
	danmar#3 ValueFlow::setValues (...) at build/valueflow.cpp:8403
	danmar#4 Tokenizer::simplifyTokens1 (...) at build/tokenize.cpp:11095
	danmar#5 CppCheck::checkFile (...) at build/cppcheck.cpp:513
	danmar#6 CppCheck::check (...) at /usr/include/c++/8/bits/basic_string.h:936
	danmar#7 CppCheckExecutor::check_internal (...) at cli/cppcheckexecutor.cpp:884
	danmar#8 CppCheckExecutor::check (...) at cli/cppcheckexecutor.cpp:198
	danmar#9 main (...) at cli/main.cpp:95

	Packages: broker
	valueFlowReverse (tokenlist=tokenlist@entry=0x7fffffffbce0, tok=tok@entry=0x555555cc9930, varToken=varToken@entry=0x555555cc9b70, val=..., val2=..., errorLogger=errorLogger@entry=0x7fffffffcb20, settings=0x7fffffffcd40) at build/valueflow.cpp:3775
	3775	                if (!assignTok->hasKnownValue()) {
	#0 valueFlowReverse (...) at build/valueflow.cpp:3775
	#1 valueFlowBeforeCondition (...) at build/valueflow.cpp:4092
	danmar#2 ValueFlow::setValues (...) at build/valueflow.cpp:8406
	danmar#3 Tokenizer::simplifyTokens1 (...) at build/tokenize.cpp:11095
	danmar#4 CppCheck::checkFile (...) at build/cppcheck.cpp:513
	danmar#5 CppCheck::check (...) at build/cppcheck.cpp:197
	danmar#6 CppCheckExecutor::check_internal (...) at cli/cppcheckexecutor.cpp:884
	danmar#7 CppCheckExecutor::check (...) at cli/cppcheckexecutor.cpp:198
	danmar#8 main (...) at cli/main.cpp:95
@rikardfalkeborn rikardfalkeborn force-pushed the donate-cpu-server-crash-locations branch from a983159 to faf95ea Compare October 17, 2019 21:56
@versat
Copy link
Copy Markdown
Collaborator

versat commented Oct 18, 2019

The improved crash report is great.
I have not found issues while testing it.
The only things I have found are some PEP8 messages regarding the regular expression.
I'll add a comment there. It is nothing that has to be fixed, but nice to change I guess.

Co-Authored-By: Sebastian <versat@users.noreply.github.com>
@rikardfalkeborn
Copy link
Copy Markdown
Contributor Author

Thanks for testing, I applied your suggestion about the regex (tested locally).

@versat versat merged commit 6e1c7e4 into danmar:master Oct 18, 2019
@rikardfalkeborn rikardfalkeborn deleted the donate-cpu-server-crash-locations branch October 18, 2019 08:42
jubnzv pushed a commit to jubnzv/cppcheck that referenced this pull request Nov 13, 2019
donate-cpu-server: Print grouped stack traces in crash report

Example output:

	Packages: psi xenomai
	Token::hasKnownValue (this=0x0) at lib/token.h:988
	988	        return mImpl->mValues && std::any_of(mImpl->mValues->begin(), mImpl->mValues->end(), std::mem_fn(&ValueFlow::Value::isKnown));
	#0 Token::hasKnownValue (...) at lib/token.h:988
	#1 valueFlowReverse (...) at build/valueflow.cpp:3775
	danmar#2 valueFlowBeforeCondition (...) at /usr/include/c++/8/bits/stl_list.h:301
	danmar#3 ValueFlow::setValues (...) at build/valueflow.cpp:8403
	danmar#4 Tokenizer::simplifyTokens1 (...) at build/tokenize.cpp:11095
	danmar#5 CppCheck::checkFile (...) at build/cppcheck.cpp:513
	danmar#6 CppCheck::check (...) at /usr/include/c++/8/bits/basic_string.h:936
	danmar#7 CppCheckExecutor::check_internal (...) at cli/cppcheckexecutor.cpp:884
	danmar#8 CppCheckExecutor::check (...) at cli/cppcheckexecutor.cpp:198
	danmar#9 main (...) at cli/main.cpp:95

	Packages: broker
	valueFlowReverse (tokenlist=tokenlist@entry=0x7fffffffbce0, tok=tok@entry=0x555555cc9930, varToken=varToken@entry=0x555555cc9b70, val=..., val2=..., errorLogger=errorLogger@entry=0x7fffffffcb20, settings=0x7fffffffcd40) at build/valueflow.cpp:3775
	3775	                if (!assignTok->hasKnownValue()) {
	#0 valueFlowReverse (...) at build/valueflow.cpp:3775
	#1 valueFlowBeforeCondition (...) at build/valueflow.cpp:4092
	danmar#2 ValueFlow::setValues (...) at build/valueflow.cpp:8406
	danmar#3 Tokenizer::simplifyTokens1 (...) at build/tokenize.cpp:11095
	danmar#4 CppCheck::checkFile (...) at build/cppcheck.cpp:513
	danmar#5 CppCheck::check (...) at build/cppcheck.cpp:197
	danmar#6 CppCheckExecutor::check_internal (...) at cli/cppcheckexecutor.cpp:884
	danmar#7 CppCheckExecutor::check (...) at cli/cppcheckexecutor.cpp:198
	danmar#8 main (...) at cli/main.cpp:95
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

X Tutup