Discussion:
[x86, irq] BUG: unable to handle kernel
Fengguang Wu
2014-10-11 02:21:05 UTC
Permalink
Greetings,

0day kernel testing robot got the below dmesg and the first bad commit is

https://github.com/jiangliu/linux.git irqdomain/p3v1
commit f0cff632d14bd3c01a8ec7ad0d5fe4f7cafa88fe
Author: Jiang Liu <***@linux.intel.com>
AuthorDate: Tue Aug 5 16:53:41 2014 +0800
Commit: Jiang Liu <***@linux.intel.com>
CommitDate: Fri Sep 26 22:06:20 2014 +0800

x86, irq: Convert IOAPIC to use hierarchy irqdomain interfaces

Convert IOAPIC driver to support and use hierarchy irqdomain interfaces.
It's a little big, but it always break bisectings if we split it into
multiple patches.

Signed-off-by: Jiang Liu <***@linux.intel.com>

+------------------------------------------+------------+------------+------------+
| | 56a45d7013 | f0cff632d1 | a1a76987cc |
+------------------------------------------+------------+------------+------------+
| boot_successes | 1000 | 277 | 6 |
| boot_failures | 0 | 398 | 6 |
| BUG:unable_to_handle_kernel | 0 | 398 | 6 |
| Oops | 0 | 398 | 6 |
| RIP:kset_find_obj | 0 | 398 | 6 |
| Kernel_panic-not_syncing:Fatal_exception | 0 | 398 | 6 |
| backtrace:__pci_register_driver | 0 | 398 | 6 |
| backtrace:imsttfb_init | 0 | 398 | 6 |
| backtrace:kernel_init_freeable | 0 | 398 | 6 |
+------------------------------------------+------------+------------+------------+

[ 2.690806] crc32c_combine: 8373 self tests passed
[ 2.710182] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
[ 2.710182] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
[ 2.731436] BUG: unable to handle kernel
[ 2.731436] BUG: unable to handle kernel paging requestpaging request at fffffffffffffff8
at fffffffffffffff8
[ 2.735257] IP:
[ 2.735257] IP: [<ffffffff811734fe>] kset_find_obj+0x2e/0x7e
[<ffffffff811734fe>] kset_find_obj+0x2e/0x7e
[ 2.735257] PGD 1a16067
[ 2.735257] PGD 1a16067 PUD 1a18067 PUD 1a18067 PMD 0 PMD 0

[ 2.735257] Oops: 0000 [#1]
[ 2.735257] Oops: 0000 [#1] DEBUG_PAGEALLOCDEBUG_PAGEALLOC

[ 2.735257] Modules linked in:
[ 2.735257] Modules linked in:

[ 2.735257] CPU: 0 PID: 1 Comm: swapper Not tainted 3.17.0-rc6-gf0cff63 #1431
[ 2.735257] CPU: 0 PID: 1 Comm: swapper Not tainted 3.17.0-rc6-gf0cff63 #1431
[ 2.735257] task: ffff880000040000 ti: ffff880000044000 task.ti: ffff880000044000
[ 2.735257] task: ffff880000040000 ti: ffff880000044000 task.ti: ffff880000044000
[ 2.735257] RIP: 0010:[<ffffffff811734fe>]
[ 2.735257] RIP: 0010:[<ffffffff811734fe>] [<ffffffff811734fe>] kset_find_obj+0x2e/0x7e
[<ffffffff811734fe>] kset_find_obj+0x2e/0x7e
[ 2.735257] RSP: 0000:ffff880000047db8 EFLAGS: 00010286
[ 2.735257] RSP: 0000:ffff880000047db8 EFLAGS: 00010286
[ 2.735257] RAX: 0000000000000000 RBX: fffffffffffffff8 RCX: ffff880000040730
[ 2.735257] RAX: 0000000000000000 RBX: fffffffffffffff8 RCX: ffff880000040730
[ 2.735257] RDX: ffff880000040066 RSI: ffffffff818ccadb RDI: 0000000000000000
[ 2.735257] RDX: ffff880000040066 RSI: ffffffff818ccadb RDI: 0000000000000000
[ 2.735257] RBP: ffff880000047dd8 R08: 0000000000000001 R09: 000000000000005f
[ 2.735257] RBP: ffff880000047dd8 R08: 0000000000000001 R09: 000000000000005f
[ 2.735257] R10: ffff880000047d38 R11: ffff88000012b940 R12: ffff88000012b928
[ 2.735257] R10: ffff880000047d38 R11: ffff88000012b940 R12: ffff88000012b928
[ 2.735257] R13: ffff88000012b918 R14: ffffffff818ccadb R15: 0000000000000000
[ 2.735257] R13: ffff88000012b918 R14: ffffffff818ccadb R15: 0000000000000000
[ 2.735257] FS: 0000000000000000(0000) GS:ffffffff81a27000(0000) knlGS:0000000000000000
[ 2.735257] FS: 0000000000000000(0000) GS:ffffffff81a27000(0000) knlGS:0000000000000000
[ 2.735257] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 2.735257] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 2.735257] CR2: fffffffffffffff8 CR3: 0000000001a15000 CR4: 00000000000006b0
[ 2.735257] CR2: fffffffffffffff8 CR3: 0000000001a15000 CR4: 00000000000006b0
[ 2.735257] Stack:
[ 2.735257] Stack:
[ 2.735257] 0000000000000050
[ 2.735257] 0000000000000050 ffffffff81c15268 ffffffff81c15268 ffff88001243db58 ffff88001243db58 ffffffff81c13500 ffffffff81c13500

[ 2.735257] ffff880000047e00
[ 2.735257] ffff880000047e00 ffffffff812dbb66 ffffffff812dbb66 ffffffff81c15268 ffffffff81c15268 0000000000000000 0000000000000000

[ 2.735257] ffff88001243db58
[ 2.735257] ffff88001243db58 ffff880000047e30 ffff880000047e30 ffffffff812cfbac ffffffff812cfbac ffffffff81c15268 ffffffff81c15268

[ 2.735257] Call Trace:
[ 2.735257] Call Trace:
[ 2.735257] [<ffffffff812dbb66>] module_add_driver+0x37/0xcc
[ 2.735257] [<ffffffff812dbb66>] module_add_driver+0x37/0xcc
[ 2.735257] [<ffffffff812cfbac>] bus_add_driver+0xf3/0x1e7
[ 2.735257] [<ffffffff812cfbac>] bus_add_driver+0xf3/0x1e7
[ 2.735257] [<ffffffff812d0ce4>] driver_register+0x89/0xc5
[ 2.735257] [<ffffffff812d0ce4>] driver_register+0x89/0xc5
[ 2.735257] [<ffffffff81198c9d>] __pci_register_driver+0x5b/0x5e
[ 2.735257] [<ffffffff81198c9d>] __pci_register_driver+0x5b/0x5e
[ 2.735257] [<ffffffff81cf0776>] ? i740fb_init+0x79/0x79
[ 2.735257] [<ffffffff81cf0776>] ? i740fb_init+0x79/0x79
[ 2.735257] [<ffffffff81cf07c8>] imsttfb_init+0x52/0xef
[ 2.735257] [<ffffffff81cf07c8>] imsttfb_init+0x52/0xef
[ 2.735257] [<ffffffff81cf0776>] ? i740fb_init+0x79/0x79
[ 2.735257] [<ffffffff81cf0776>] ? i740fb_init+0x79/0x79
[ 2.735257] [<ffffffff810003b8>] do_one_initcall+0x19b/0x1b0
[ 2.735257] [<ffffffff810003b8>] do_one_initcall+0x19b/0x1b0
[ 2.735257] [<ffffffff81077599>] ? parse_args+0x22c/0x360
[ 2.735257] [<ffffffff81077599>] ? parse_args+0x22c/0x360
[ 2.735257] [<ffffffff81cd5f4b>] kernel_init_freeable+0xf7/0x17a
[ 2.735257] [<ffffffff81cd5f4b>] kernel_init_freeable+0xf7/0x17a
[ 2.735257] [<ffffffff81581758>] ? rest_init+0xbc/0xbc
[ 2.735257] [<ffffffff81581758>] ? rest_init+0xbc/0xbc
[ 2.735257] [<ffffffff81581761>] kernel_init+0x9/0xd5
[ 2.735257] [<ffffffff81581761>] kernel_init+0x9/0xd5
[ 2.735257] [<ffffffff8159047a>] ret_from_fork+0x7a/0xb0
[ 2.735257] [<ffffffff8159047a>] ret_from_fork+0x7a/0xb0
[ 2.735257] [<ffffffff81581758>] ? rest_init+0xbc/0xbc
[ 2.735257] [<ffffffff81581758>] ? rest_init+0xbc/0xbc
[ 2.735257] Code:
[ 2.735257] Code: e5 e5 41 41 56 56 41 41 55 55 41 41 54 54 4c 4c 8d 8d 67 67 10 10 53 53 49 49 89 89 fd fd 49 49 89 89 f6 f6 4c 4c 89 89 e7 e7 e8 e8 32 32 c2 c2 41 41 00 00 49 49 8b 8b 45 45 00 00 48 48 8d 8d 58 58 f8 f8 48 48 8d 8d 43 43 08 08 49 49 39 39 c5 c5 74 74 39 39 <48> <48> 8b 8b 3b 3b 48 48 85 85 ff ff 74 74 27 27 4c 4c 89 89 f6 f6 e8 e8 39 39 4a 4a 00 00 00 00 85 85 c0 c0 75 75 1b 1b 8b 8b

[ 2.735257] RIP
[ 2.735257] RIP [<ffffffff811734fe>] kset_find_obj+0x2e/0x7e
[<ffffffff811734fe>] kset_find_obj+0x2e/0x7e
[ 2.735257] RSP <ffff880000047db8>
[ 2.735257] RSP <ffff880000047db8>
[ 2.735257] CR2: fffffffffffffff8
[ 2.735257] CR2: fffffffffffffff8
[ 2.735257] ---[ end trace 149a66b8c5f660be ]---
[ 2.735257] ---[ end trace 149a66b8c5f660be ]---

git bisect start a1a76987cc9d7914f4a7c4f6603be16fc4a96306 bfe01a5ba2490f299e1d2d5508cbbbadd897bbe9 --
git bisect good 5978adbd3eed4e876e6841e9cc35da05efac529b # 23:32 675+ 0 x86: irq_remapping: Introduce new interfaces to support hierarchy irqdomain
git bisect bad 4eb4d80e3b0556e6b678307731d19454934b71b7 # 23:32 29- 1 x86, irq: Kill x86_io_apic_ops.print_entries and related interfaces
git bisect good 0a6a0c523fc095099c5a73fe05d29558aa50d3f9 # 23:41 675+ 0 x86, irq: Clean up unused MSI related code and interfaces
git bisect good cbbc1e9fd24c705d5b958d4b09e75e9da6461857 # 23:50 675+ 0 x86, irq: Kill unused setup_timer_IRQ0_pin()
git bisect good 56a45d70134ae72f85c87fb77caf571968907854 # 00:03 675+ 0 x86, irq: Refine the way to allocate irq_cfg for legacy IRQs
git bisect bad 27465585446f65cb231c4866e8cde1e1dcdfb31b # 00:12 37- 307 x86, irq: Kill unused old IOAPIC irqdomain interfaces
git bisect bad f0cff632d14bd3c01a8ec7ad0d5fe4f7cafa88fe # 00:23 41- 69 x86, irq: Convert IOAPIC to use hierarchy irqdomain interfaces
# first bad commit: [f0cff632d14bd3c01a8ec7ad0d5fe4f7cafa88fe] x86, irq: Convert IOAPIC to use hierarchy irqdomain interfaces
git bisect good 56a45d70134ae72f85c87fb77caf571968907854 # 00:35 1000+ 0 x86, irq: Refine the way to allocate irq_cfg for legacy IRQs
git bisect bad a1a76987cc9d7914f4a7c4f6603be16fc4a96306 # 00:35 0- 6 Merge remote-tracking branch 'jiangliu/irqdomain/p3v1' into for-suravee
git bisect good c798360cd1438090d51eeaa8e67985da11362eba # 00:43 1000+ 0 Merge branch 'for-3.18' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/percpu
git bisect good 22f3190416445059ec17332efd71e78ec969d37f # 00:52 1000+ 0 Add linux-next specific files for 20141010


This script may reproduce the error.

----------------------------------------------------------------------------
#!/bin/bash

kernel=$1
initrd=yocto-minimal-x86_64.cgz

wget --no-clobber https://github.com/fengguang/reproduce-kernel-bug/raw/master/initrd/$initrd

kvm=(
qemu-system-x86_64
-cpu kvm64
-enable-kvm
-kernel $kernel
-initrd $initrd
-m 320
-smp 1
-net nic,vlan=1,model=e1000
-net user,vlan=1
-boot order=nc
-no-reboot
-watchdog i6300esb
-rtc base=localtime
-serial stdio
-display none
-monitor null
)

append=(
hung_task_panic=1
earlyprintk=ttyS0,115200
debug
apic=debug
sysrq_always_enabled
rcupdate.rcu_cpu_stall_timeout=100
panic=-1
softlockup_panic=1
nmi_watchdog=panic
oops=panic
load_ramdisk=2
prompt_ramdisk=0
console=ttyS0,115200
console=tty0
vga=normal
root=/dev/ram0
rw
drbd.minor_count=8
)

"${kvm[@]}" --append "${append[*]}"
----------------------------------------------------------------------------

Thanks,
Fengguang

Loading...