cloud-init和rc.local

What is cloud-init?

Cloud-init is a service that customizes Linux-based cloud servers. It's supported by most Linux distributions and public cloud vendors, and it's the industry standard to initialize cloud virtual machines. Canonical created cloud-init more than a decade ago for Ubuntu, and since then, all major public clouds and most Linux distributions have added support for cloud-init. 

Advantages of cloud-init

You'll discover several advantages when you start using cloud-init. First, it's portable. If you developed scripts at another public cloud, you could deploy those at Vultr without changes. You can also do many things with cloud-init that aren't easy to do with imageboot scripts, like:

  • You can override Vultr's vendor-data with your user-data.
  • You can inject shell scripts that run once per boot or once per instance.
  • You have advanced logging and debugging tools with cloud-init analyze.
  • It's easy to reset an instance and run cloud-init again without a full reinstall.

What is cloud-init?

Cloud-init is an industry standard method for automating cloud instance initialization, with support across distributions and platforms. Cloud-init manages initialization using a combination of instance metadata and configuration scripts (user data) to automate the process of setting up a new server.

Akamai’s Metadata service provides an API for cloud-init to consume, offering the relevant instance and user data to initialize your server. When your new instance spins up, cloud-init starts running locally, accesses the metadata, and automatically configures your system accordingly.

Metadata Availability
Akamai’s Metadata service is now available in select data centers. Use Metadata to automate system configuration by adding directives or scripts when deploying Compute Instances. This user data can then be consumed by cloud-init, an industry standard system initialization tool, or accessed directly using the Metadata API. For instructions on using the Metadata service and for a list of supported regions and distributions, reference our documentation.

Differences:

cloud-init and rc.local are both methods for initializing and configuring a Linux system during the boot process. However, they serve different purposes and have distinct characteristics.

cloud-init:

cloud-init is a cloud-based initialization framework specifically designed for provisioning and configuring virtual machines (VMs) in cloud environments. It provides a standardized and automated approach to setting up cloud instances, including tasks like:

  1. Network Configuration: Setting up the network interface and assigning IP addresses based on cloud metadata.

  2. User Configuration: Creating user accounts and setting passwords based on cloud-provided credentials.

  3. Package Installation: Installing software packages and configuring services according to user-defined scripts or cloud metadata.

  4. Run Commands: Executing arbitrary commands during the boot process to perform specific system customizations.

cloud-init typically runs early in the boot process, before the system fully enters the multi-user runlevel. It utilizes cloud-provided metadata or user-supplied configuration files to determine the actions to be performed.

rc.local:

rc.local is a traditional script file used for executing custom commands or scripts during the boot process. It's located at /etc/rc.local and is typically executed at the end of the boot process, after the system has entered the multi-user runlevel.

rc.local is a more general-purpose tool compared to cloud-init. It can be used for various tasks, such as:

  1. Starting System Services: Launching specific services that should start after the system has initialized.

  2. Mounting Filesystems: Mounting additional filesystems that are not automatically mounted by the system.

  3. Running User Scripts: Executing user-defined scripts to perform specific configuration or customization tasks.

rc.local's execution order is determined by its filename and placement within the /etc/rc.d directory structure.

Key Differences:

Featurecloud-initrc.local
PurposeCloud-based VM provisioning and configurationGeneral-purpose boot-time script execution
Execution TimingEarly in the boot processLate in the boot process
Configuration SourceCloud-provided metadata or user-supplied configuration filesUser-defined scripts or commands
Typical Use CasesNetwork configuration, user configuration, package installation, command executionStarting services, mounting filesystems, running user scripts

Choosing Between cloud-init and rc.local:

  • Use cloud-init: If you're managing VMs in a cloud environment, cloud-init is the preferred choice for its automation, standardization, and cloud-integration capabilities.

  • Use rc.local: For specific tasks that require execution after the system has initialized and for more granular control over boot-time scripts, rc.local is a suitable option.

In summary, cloud-init is a powerful tool specifically designed for cloud-based VM provisioning and configuration, while rc.local is a more general-purpose script execution mechanism for boot-time tasks. The choice between the two depends on the specific needs and environment of your system.

[root@vultr ~]# cat /etc/cloud/cloud.cfg
cloud_init_modules:
  - ssh
  - yum-add-repo
cloud_config_modules:
  - ssh-import-id
  - keyboard
cloud_final_modules:
  - ansible
  - mcollective
  - salt-minion
system_info:
  distro: centos
  paths:
    cloud_dir: /var/lib/cloud/
    templates_dir: /etc/cloud/templates/
  ssh_svcname: sshd

Troubleshoot metadata and cloud-init

View cloud-init logs

If you encounter issues with cloud-init or your user data, you can review the logs and output logs by running one of the commands below:

cat /var/log/cloud-init.log
cat /var/log/cloud-init-output.log

If you are not able to access your system through SSH, you can use Lish or boot your instance into rescue mode and mount your disks.

Consider Using Akamai's New Metadata Service

Akamai's Metadata service is now available in select data centers and should be considered for new projects as an alternative to StackScripts. Use Metadata to automate system configuration by adding directives or scripts when deploying Compute Instances. This user data can then be consumed by cloud-init, an industry standard system initialization tool, or accessed directly using the Metadata API. For instructions on using the Metadata service and for a list of supported regions and distributions, reference our documentation.

User data formats

User data can be provided in many different formats, with the most common being cloud-config.

  • Cloud-config script: cloud-config is the default syntax for cloud-init and can be used on any Linux distribution. It contains a list of directives formatted using YAML. Review the Using cloud-config files to configure a server guide for more details.

    #cloud-config
package_update: true
package_upgrade: true
packages:
- nginx
- mysql-server

  • Executable script: Cloud-init also accepts other scripts that can be executed by the target distribution. This includes bash and python. Since many commands (including those to create users and install packages) differ between distributions, providing these scripts may limit which distributions you can target.

    #!/bin/bash
apt-get update -y && apt-get upgrade -y
apt-get install nginx mysql-server -y

  • Other formats: Review the User data formats guide within the official documentation to learn more about other types of formats supported by cloud-init.

  • 测试：

[root@vultr ~]# cat test.yaml 
#cloud-config
timezone: Asia/Shanghai
[root@vultr ~]# cloud-init schema --config test.yaml
Valid schema test.yaml
[root@vultr ~]# 
[root@vultr ~]# cat test2.yaml 
#!/bin/sh
echo "Hello World" > /var/tmp/output.txt
[root@vultr ~]# cloud-init schema --config test2.yaml
User-data type 'text/x-shellscript' not currently evaluated by cloud-init schema
[root@172-236-13-22 ~]# cat test3.yaml 
#cloud-config
timezone: Asia/Shanghai
#!/bin/sh
echo "Hello World" > /var/tmp/output.txt
[root@172-236-13-22 ~]# cloud-init schema --config test3.yaml 
Invalid UNKNOWN_CONFIG_HEADER test3.yaml
Error: Cloud config schema errors: format-l4.c1: File test3.yaml is not valid YAML. while scanning a simple key
  in "<unicode string>", line 4, column 1:
    echo "Hello World" > /var/tmp/ou ... 
    ^
could not find expected ':'
  in "<unicode string>", line 5, column 1:
    
    ^

Error: Invalid schema: UNKNOWN_CONFIG_HEADER
使用MIME multi-part：
vbird@DESKTOP-3NQUT0E:~$ cloud-init devel make-mime -a set-tz-cloud-config.txt:cloud-config -a install_ss.sh:x-shellscript > userdata3
vbird@DESKTOP-3NQUT0E:~$ cat userdata3
Content-Type: multipart/mixed; boundary="===============3740234081866345669=="
MIME-Version: 1.0

--===============3740234081866345669==
Content-Type: text/cloud-config; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="set-tz-cloud-config.txt"

I2Nsb3VkLWNvbmZpZwp0aW1lem9uZTogQXNpYS9TaGFuZ2hhaQo=

--===============3740234081866345669==
Content-Type: text/x-shellscript; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="install_ss.sh"

IyEvYmluL2Jhc2gKIyAxCnN5c3RlbWN0bCBzdG9wIGZpcmV3YWxsZAplY2hvICJcbiMjIyMjI1xu

评论

发表评论

此博客中的热门博文

码率单位

Feature Bandwidth Traffic Throughput Definition Maximum data transfer rate Actual data being transmitted Actual data transfer rate achieved Measure Capacity (bps) Data volume (bytes or packets) Transfer rate (bps) Static vs. Dynamic Static (connection property) Dynamic (varies over time) Dynamic (affected by network conditions) Impact on Performance Determines potential data transfer speed Affects network congestion and utilization Measures actual network performance 表格一览： 名称 符号 b/s B/s 示例 比特每秒 b/s 1 1/8 字节每秒 B/s 8 1 千比特每秒 kb/s 1000 = 10^3 125 = 10^3/8 二进制千比特每秒 Kib/s 1024 = 2^10 128 = 2^10/8 千字节每秒 kB/s 8000 = 8*10^3 1000 = 10^3 二进制千字节每秒 KiB/s 8192 = 8*2^10 1024 = 2^10 k 与 Ki k 与 Ki 分别表示 kilo-（千） 与 kibi-（二进制千） 。作为前缀使用时， k 表示 1,000，Ki 表示1,024 ，因为“Ki”来源于它在计算机方面 2^10 = 1,024 的使用。不幸的是，人们常常错误地用 K 代替 Ki。此外，不知晓其中微妙差别的广大公众，常常不加区别地使用“Kbps”与“Kibps”，造成了混乱。总之，使用“Kibps”通常是正确的。 The bit rate is expressed in the unit bit per second (symbol: bit/s), often in conjunction with an SI prefix such as kilo (1...
阅读全文

Process vs. Thread

图片
  A process is an executing instance of a program with its own memory and resources, while  a thread is the smallest unit of execution within a process that can run concurrently with other threads in the same process .  Processes are isolated with separate address spaces and require system-provided inter-process communication to share data, whereas threads share the process's memory and resources, allowing for faster communication and context switching.   Process vs. Thread Process: An independent program in execution.  Has its own separate memory address space.  Carries more state information and resources (like open files and system handles).  Data sharing requires Inter-Process Communication (IPC) mechanisms.  Considered a "heavyweight" unit of execution due to resource overhead for creation and context switching.  Thread: A subset of a process , executing within the process's context.  Shares the process's address space, memory, and ...
阅读全文

日志文件系统

图片
Journaling filesystem 基本原理 更新文件系统以反映文件和目录的更改通常需要许多单独的写入操作。这使得写入之间可能发生中断（例如电源故障或系统崩溃），从而使数据结构处于无效的中间状态。[1] 例如，删除 Unix 文件系统上的文件涉及三个步骤：[7] 删除其目录条目。（Removing its directory entry.） 将索引节点释放到空闲索引节点池中。（Releasing the inode to the pool of free inodes.） 将所有磁盘块返回到空闲磁盘块池。（Returning all disk blocks to the pool of free disk blocks.） 如果在步骤1之后和步骤2之前发生崩溃，就会出现孤立的inode，从而导致存储泄漏；如果在步骤2和步骤3之间发生崩溃，则该文件先前使用的块将无法用于新文件，从而有效地减少了文件系统的存储容量。重新安排步骤也无济于事。如果步骤 3 在步骤 1 之前，则它们之间的崩溃可能允许文件的块重新用于新文件，这意味着部分删除的文件将包含另一个文件的部分内容，并且对任一文件的修改都会显示在两个文件中。另一方面，如果步骤 2 在步骤 1 之前，则它们之间的崩溃将导致文件无法访问，尽管该文件看起来存在。 检测此类不一致并从中恢复通常需要完整地检查其数据结构，例如通过 fsck （文件系统检查器）等工具。[2] 这通常必须在下次安装文件系统以进行读写访问之前完成。如果文件系统很大并且 I/O 带宽相对较小，则这可能会花费很长时间，并且如果它阻止系统的其余部分恢复联机，则会导致更长的停机时间。 为了防止这种情况，日志文件系统分配一个特殊区域——日志——在其中记录它将提前进行的更改。崩溃后，恢复只需从文件系统读取日志并重播该日志中的更改，直到文件系统再次保持一致。因此，这些更改被认为是原子的（不可整除），因为它们要么成功（最初成功或在恢复期间完全重播），要么根本不重播（被跳过，因为它们在之前尚未完全写入日志）发生崩溃）。  JFS(1990)->NTFS(1993)->XFS(1994)->Apple HFS Plus(1998)->ext3(2001)->ext4(2008) ext和ext2文件系统不是日志文件系统 日誌式檔...
阅读全文