Speed up boot through disk sorting

From WorkOutWiki2008

Contents 1 Proposer 2 Purpose 3 Abstract 4 Pre-requisites 4.1 Languages, tools to be known 4.2 Getting and compiling the code 4.3 Links to overall design/architecture 5 Pre-event planning 5.1 Tasks 5.1.1 Knowledge Reqd: Algorithms and data structures, C, Linux system calls and understanding of the problem ;) 5.1.2 Knowledge Reqd: Device mapper, lvm 5.2 Existing work 5.3 Getting in touch 6 Participants 6.1 Progress

Proposer

Nikanth Karthikesan Suse Labs, Novell

Purpose

Using device mapper to optimize the disk layout for a certain workload to minimize seeks. The goal of this workout is to do this for bootup as well.

Abstract

I have already experimented with recording the disk access patterns and relocating the disk blocks below the file-system using device mapper to avoid seeks, which can speed up things on hard-disks. The solution uses blktrace to capture the access pattern and later computes the optimal disk layout and then moves around the disk blocks and sets up device mapper tables to speed up the same work-load.

The goal of this workout is to do this for bootup as well. Currently this works on bare device mapper level, hence the meta data handling is difficult and booting from such a disk is unsupported. Make this on top of lvm2 to add boot support.

Pre-requisites

Laptops with blkparse & device-mapper installed. Shouldn't be a problem if Internet is available. Laptops with root on LVM for boot optimization. People with spare hard-disk either internal/external would be welcome.

blktrace, device-mapper, lvm2 And possibly some use-cases what they want to optimize. These are optional because even without knowledge of these tools, the algorithm can be improved.

Languages, tools to be known

• Mandatory: C
• Optional: device mapper, lvm2, blktrace, seekwatcher

Getting and compiling the code

http://gitorious.org/projects/disk-sort

Links to overall design/architecture

http://gitorious.org/projects/disk-sort/repos/mainline/blobs/master/README

This is inspired by Fcache But extends the concept beyond boot and applies it for any workloads.

Pre-event planning

Tasks

Knowledge Reqd: Algorithms and data structures, C, Linux system calls and understanding of the problem ;)

• Optimize/change the algorithms used
• Change the programs or start from scratch to do it faster and better [currently uses sqlite – but would be better to do it in memory]
• Add support for considering seek distance into account
• Minimize the disk movement required
• Better undo support

Knowledge Reqd: Device mapper, lvm

• Integrate with lvm2 (ftp://sources.redhat.com/pub/lvm2/). Required for boot optimization.
• Write a easy-to-use wrapper to automate everything

And whatever that comes out of the discussion

Existing work

Current state – just a prototype

• Couple of tools
  • to convert the blktrace output to an ideal layout
  • to shuffle the blocks in the disk to this new “ideal” layout

Getting in touch

knikanth =at= suse =dot= de

Participants

Nikanth Karthikesan:

• Conside seek distances when calculating ideal layout
• lvm2 integration.

To add yourself here mail me at the above email address.

Progress

Speed_up_boot_through_disk_sorting_Workout_Report