Development Tools Microconference Notes




Welcome to Linux Plumbers Conference 2015



The structure will be short introductions to an issue or topic followed by a discussion with the audience.


A limit of 3 slides per presentation is enforced to ensure focus and allocate enough time for discussions.



Please use this etherpad to take notes. Microconf leaders will be giving a TWO MINUTE summary of their microconference during the Friday afternoon closing session.



Please remember there is no video this year, so your notes are the only record of your microconference.



Miniconf leaders: Please remember to take note of the approximate number of attendees in your session(s).


  1. Wifi SSID: Exhibitor Internet Login: linux Password: seattle


SCHEDULE





·         Speaker: Shuah Khan from Samsung.

·         Abstract: https://linuxplumbersconf.org/2015/ocw/proposals/2643

·         This is a 20 minute talk

·         Debugging methods

o    Redirecting console

·         Net console has limited use

o    Enable early printk right away to get messages from early boot sequence

o    Assuming system boots up, but you want to see if system state is good, can
poke with lots of tools

·         Lspci, lsusb to look at devices

o    Panics:

·         Oops messages are helpful

o    System hangs - SysRq key helps you get out of the state to see debug info

o    Races/timing:

·         Note what device, what usage

o    e.g. Video application tuning into TV channel & see streaming errors

·         Are there changes in device driver?

·         Are there new features added?

·         What are the conditions?

·         Debugging tools

o    How do you look into oops message?

·         Use GDB to investigate

o    KDB needs to be configured

·         She doesn't use it so much. More GDB

·         Low level debugger that can poke at addresses directly

·         KGDB sits on top of KDB to do more complex things

o    Kernel address sanitizer

·         There was a talk yesterday on it

·         It's a new tool added to 4.0 kernel that will find memory errors

·         Will find things like memory use after free

·         Do you do ARM & x86?

o    Work on both

o    ARM-64 kernel was available before KGDB

·         Linaro has done some back-ports so that the major platforms do have
back-ends in the kernel

·         Kernel Debug interfaces

o    Tend to user in-kernel debug interfaces when debugging drivers

·         e.g. for DMA will enable DMA debug

o    Kernel has a lot of kernel debug config options

·         Grep on your config file & see what are turned on by default

·         Downside is that these are compile-time so may need to re-compile to
have them

§  Your race condition might not reproduce if you re-compile

o    If you're adding a new driver, you want to enable debug options up front

o    Dynamic debug can be enabled selectively to trigger print message or debug
message for a specific module

·         She does it with her x86 system with Intel MEI driver when needs to
debug that

·         Good is that you can trigger on per-module basis or specific debug
message, if you are anticipating a specific event code or init code

·         Remember to make it persistent if you want it to persist across
reboots. Default is not persistent.

o    Tracepoints

·         Even with dynamic debug, you have the code already in there

·         Advantage of tracepoints is that you don't incur the overhead of
having the code compiled in. It will be like code branch modification.

·         However, don't have as wide trace points support across the kernel

·         She will use this heavily when debugging IOMMU type issues & likes to
use it when it is available

·         Can enable/disable at runtime, so don't need to recompile or install
new image.

·         Can do stuff like find out whenever when move from host to VM & VM to
host

·         Can trigger on a particular event

·         This helps to give more granularity instead of just sifting through a
ton of messages

·         Git bisect to isolate bad commit

o    She uses this a lot

·         Kernel debug configuration options

o    Favourite: Lock debugging when starting on new work to get a baseline for
what issues already exist

o    Some of these are needed for kmemcheck to work

·         Complaint that enabling some debug makes life worse

o    Sometimes debug code has bugs too…

o    Make sure your code runs both with & without a certain option enabled

·         Kmemcheck

o    She gives some options that you need

·         Kernel Address Sanitizer

o    There are other sanitizers too: Thread sanitizer, user space tools (are
quite mature), now moving towards compile-time instrumentation


·         Abstract: https://linuxplumbersconf.org/2015/ocw/proposals/2673

·         Speaker: Mike Godfrey, professor at Uwaterloo (Software analytics)

·         Doesn't write code, but grinds it up & analyzes it

·         All we like sheep - from Handel's Messiah. Sheep = Dolly

·         What is software clones & clone detection?  (audience game)

o    Example 1: Why are there 2 different copies of the startup routine in
different files?

o    Example 2: This is a more reasonable case of nearly identical conversion
routines, but they are unlikely to vary from each other & they are all in the
same file

o    Example 3: const pointer vs. pointer to const fixed in 1 place, but not
everywhere ("Inconsistent maintenance")

·         What's a clone:

o    The tools have different assumptions - depends on the tolerances &
algorithms.  Thus need a particular definition of similarity.

·         Bellon's taxonomy

o    Lists out the different types of similarity

·         Code clone detection methods

o    Either you have a stream or a graph

o    Problem is that as you become more language dependent, it gets more
complicated

o    So, there are some other methods

·         Lightweight semantics: Look at callers & what get called only to see
what overlap you get. Then do a more precise hand-analysis.

·         Hashes are good for exact matches, but sometimes you need some
flexibility

·         When is cloning a good idea?

o    There are big authorities that say it's always bad

o    Myth: Code cloning is bad in the long run

·         Down sides

§  Code bloated

§  Ossifies

§  Inconsistent maintenance (fixing the code in all the copied places)

·         Fix it by refactoring to the good design.

·         XP's rule of 3: (XP = Extreme Programming) If you do something 3
times, you've discovered an abstraction & should spend some time on it

·         Is inconsistent maintenance really a problem?

§  Things are changed inconsistently 1/2 the time, many times they intend to
diverge

·         Is cloned code buggier?

§  Most bugs have little to do with clones. Cloned code was typically less buggy

·         Linux SCSI driver cloning example

§  Does cloning predict pcompatible bus type dependencies?

·         10% for random

·         50% for vendor

·         Even higher for cloned

·         Is cloning considered harmful?

o    Paper: "Cloning considered harmful considered harmful"

·         When you de-clone, you may lose some information

·         Sometimes you have specific design goals in mind when you do clones

·         They came up with about 10 patterns in 3 groups

§  Forking: Want to do it similarly, but somewhere else & don't want a ton of
ifdefs (e.g. Linux SCSI drivers)

·         Works well when commonalities of end solutions are unclear (e.g.
platform variation) & the cloning is obvious and well documented

§  Templating: Similar, but not similar enough to make 1 copy with parametrized
differences

§  Post-hoc customizing: their "misc" bucket

·         If you don't own the code

·         e.g. "clone & own" from Microsoft Research

·         Cloning is often useful in the long run if you understand the
technical benefits