企业官网建设流程全解析

目录

1.知识回顾

2.前置知识:fwrite函数

2.对比示例代码的运行结果

示例代码1

示例代码2:去掉示例代码1中所有字符串中的\n

示例代码3:注释掉示例代码2的close(1)

3.分析示例代码,进一步引入C语言的缓冲区

前置知识: 刷新缓冲区

初步结论

进一步分析

从内核代码看默认情况下的write将数据先写入内核缓冲区,再落磁盘

结论

得出原因

查看printf、fprintf和fwrite写入的缓冲区

FILE结构体

4.回顾exit和_exit

5.缓冲区刷新策略

C语言缓冲区刷新所有方法

1.知识回顾

之前在一些文章提到过C语言的缓冲区:

24.【C语言】getchar和putchar的使用

89.【C语言】文件操作(6)

本文进一步解释

2.前置知识:fwrite函数

函数声明为:

size_t fwrite(const void ptr[restrict .size * .nmemb], size_t size, size_t nmemb, FILE *restrict stream);

fwrite以二进制形式对文件进行操作(把ptr所指向的数组中的数据写入到给定流stream中),不局限于文本文件,可以看到ptr是指向用于写入的元素数组的指针,其类型为constvoid,那么传参给fwrite涉及到ptr的类型转换

size表示一个块的大小(单位是字节),nmemb表示要求写入的块的个数,那么一共需要写入的大小为nmemb*size,但fwrite返回的是实际写入的块数(不一定等于nmemb),可能ptr指向数组的大小是小于nmemb*size的)

stream是FILE*类型的指针

2.对比示例代码的运行结果

示例代码1

#include <stdio.h> #include <string.h> #include <unistd.h> int main() { const char* str1 = "hello printf\n"; const char* str2 = "hello fprintf\n"; const char* str3 = "hello fwrite\n"; printf("%s",str1); // stdout -> 1 fprintf(stdout, "%s",str2); // stdout -> 1 fwrite(str3, strlen(str3), 1, stdout); // fwrite, stdout -> 1 close(1); return 0; }

运行结果:正常打印所有字符串

示例代码2:去掉示例代码1中所有字符串中的\n

#include <stdio.h> #include <string.h> #include <unistd.h> int main() { const char* str1 = "hello printf"; const char* str2 = "hello fprintf"; const char* str3 = "hello fwrite"; printf("%s",str1); // stdout -> 1 fprintf(stdout, "%s",str2); // stdout -> 1 fwrite(str3, strlen(str3), 1, stdout); // fwrite, stdout -> 1 close(1); return 0; }

运行结果:什么都没有打印

示例代码3:注释掉示例代码2的close(1)

#include <stdio.h> #include <string.h> int main() { const char* str1 = "hello printf"; const char* str2 = "hello fprintf"; const char* str3 = "hello fwrite"; printf(str1); // stdout -> 1 fprintf(stdout, str2); // stdout -> 1 fwrite(str3, strlen(str3), 1, stdout); // fwrite, stdout -> 1 //close(1); return 0; }

运行结果:正常打印

3.分析示例代码,进一步引入C语言的缓冲区

前置知识: 刷新缓冲区

刷新缓冲区指的是将缓冲区的数据写入流中然后清空缓冲区,那么刷新到显示器指的是刷新缓冲区的数据到显示器

初步结论

之前讲过,进程退出之前会自动刷新缓冲区

对于示例代码2,printf、fprintf和fwrite都向缓冲区中写入不含有\n字符串,那么缓冲区不会强制刷新到显示器上,执行close(1),接着进程退出之前执行自动刷新缓冲区的操作,但没有字符串被打印出来

推出初步的结论:这个缓冲区一定不在操作系统内,是C语言自带的缓冲区,它不是操作系统(内核)的缓冲区

可以反向思考: 如果缓冲区在操作系统内部,那么调用C语言文件函数,其内部使用write系统调用将字符串写入到操作系统的缓冲区,由于close是Linux的系统调用,那么close(1)时会将字符串刷新到屏幕,但示例代码2的现象是没有字符串被打印出来

进一步分析

通过printf、fprintf和fwrite写入字符串,字符串会先暂存到C语言提供的缓冲区,等到合适的时机时再通过write将它们写入到系统的缓冲区,且显示字符串需要刷新操作系统内部的缓冲区

准确来说: 默认情况下(类似int fd = open("testfile", O_WRONLY|O_CREAT, 0644);write(fd, "hello", 5);),write先将数据写入到系统的缓冲区,待时机成熟后再将缓冲区的数据刷新到磁盘

从内核代码看默认情况下的write将数据先写入内核缓冲区,再落磁盘

以EXT4系统为例,看linux v6.19.10的write函数的调用链(读者可以去read_write.c - fs/read_write.c - Linux source code v6.19.10 - Bootlin Elixir Cross Referencer追踪验证)

write系统调用

→ksys_write

→vfs_write

→ext4的struct file(定义在/fs/ext4/file.c中)没有file->f_op->write

→new_sync_write

→ext4_file_write_iter

→没有DAX和O_DIRECT标志的情况

→ext4_buffered_write_iter

→generic_perform_write

→copy_folio_from_iter_atomic

→......

→generic_write_sync

→......

这里给出关键函数的定义:

static ssize_t ext4_buffered_write_iter(struct kiocb *iocb, struct iov_iter *from) { ssize_t ret; struct inode *inode = file_inode(iocb->ki_filp); if (iocb->ki_flags & IOCB_NOWAIT) return -EOPNOTSUPP; inode_lock(inode); ret = ext4_write_checks(iocb, from); if (ret <= 0) goto out; ret = generic_perform_write(iocb, from); out: inode_unlock(inode); if (unlikely(ret <= 0)) return ret; return generic_write_sync(iocb, ret); }

看两个函数的作用:

先看

ssize_t generic_perform_write(struct kiocb *iocb, struct iov_iter *i) { struct file *file = iocb->ki_filp; loff_t pos = iocb->ki_pos; struct address_space *mapping = file->f_mapping; const struct address_space_operations *a_ops = mapping->a_ops; size_t chunk = mapping_max_folio_size(mapping); long status = 0; ssize_t written = 0; do { struct folio *folio; size_t offset; /* Offset into folio */ size_t bytes; /* Bytes to write to folio */ size_t copied; /* Bytes copied from user */ void *fsdata = NULL; bytes = iov_iter_count(i); retry: offset = pos & (chunk - 1); bytes = min(chunk - offset, bytes); balance_dirty_pages_ratelimited(mapping); if (fatal_signal_pending(current)) { status = -EINTR; break; } status = a_ops->write_begin(iocb, mapping, pos, bytes, &folio, &fsdata); if (unlikely(status < 0)) break; offset = offset_in_folio(folio, pos); if (bytes > folio_size(folio) - offset) bytes = folio_size(folio) - offset; if (mapping_writably_mapped(mapping)) flush_dcache_folio(folio); /* * Faults here on mmap()s can recurse into arbitrary * filesystem code. Lots of locks are held that can * deadlock. Use an atomic copy to avoid deadlocking * in page fault handling. */ copied = copy_folio_from_iter_atomic(folio, offset, bytes, i); flush_dcache_folio(folio); status = a_ops->write_end(iocb, mapping, pos, bytes, copied, folio, fsdata); if (unlikely(status != copied)) { iov_iter_revert(i, copied - max(status, 0L)); if (unlikely(status < 0)) break; } cond_resched(); if (unlikely(status == 0)) { /* * A short copy made ->write_end() reject the * thing entirely. Might be memory poisoning * halfway through, might be a race with munmap, * might be severe memory pressure. */ if (chunk > PAGE_SIZE) chunk /= 2; if (copied) { bytes = copied; goto retry; } /* * 'folio' is now unlocked and faults on it can be * handled. Ensure forward progress by trying to * fault it in now. */ if (fault_in_iov_iter_readable(i, bytes) == bytes) { status = -EFAULT; break; } } else { pos += status; written += status; } } while (iov_iter_count(i)); if (!written) return status; iocb->ki_pos += written; return written; } EXPORT_SYMBOL(generic_perform_write);

generic_perform_write内部调用了write_begin函数:

status = a_ops->write_begin(iocb, mapping, pos, bytes,&folio, &fsdata);

在/Documentation/filesystems/vfs.rst里面明确提到了write_begin函数的职责"The filesystem must returnthe locked pagecache(页缓存! 说明了"generic_perform_write负责将数据写入页缓存") foliofor the
specified offset":

``write_begin`` Called by the generic buffered write code to ask the filesystem to prepare to write len bytes at the given offset in the file. The address_space should check that the write will be able to complete, by allocating space if necessary and doing any other internal housekeeping. If the write will update parts of any basic-blocks on storage, then those blocks should be pre-read (if they haven't been read already) so that the updated blocks can be written out properly. The filesystem must return the locked pagecache folio for the specified offset, in ``*foliop``, for the caller to write into. It must be able to cope with short writes (where the length passed to write_begin is greater than the number of bytes copied into the folio). A void * may be returned in fsdata, which then gets passed into write_end. Returns 0 on success; < 0 on failure (which is the error code), in which case write_end is not called.

继续看generic_write_sync:

/* * Sync the bytes written if this was a synchronous write. Expect ki_pos * to already be updated for the write, and will return either the amount * of bytes passed in, or an error if syncing the file failed. */ static inline ssize_t generic_write_sync(struct kiocb *iocb, ssize_t count) { if (iocb_is_dsync(iocb)) { int ret = vfs_fsync_range(iocb->ki_filp, iocb->ki_pos - count, iocb->ki_pos - 1, (iocb->ki_flags & IOCB_SYNC) ? 0 : 1); if (ret) return ret; } else if (iocb->ki_flags & IOCB_DONTCACHE) { struct address_space *mapping = iocb->ki_filp->f_mapping; filemap_flush_range(mapping, iocb->ki_pos - count, iocb->ki_pos - 1); } return count; } /** * vfs_fsync_range - helper to sync a range of data & metadata to disk * @file: file to sync * @start: offset in bytes of the beginning of data range to sync * @end: offset in bytes of the end of data range (inclusive) * @datasync: perform only datasync * * Write back data in range @start..@end and metadata for @file to disk. If * @datasync is set only metadata needed to access modified file data is * written. */ int vfs_fsync_range(struct file *file, loff_t start, loff_t end, int datasync) { struct inode *inode = file->f_mapping->host; if (!file->f_op->fsync) return -EINVAL; if (!datasync && (inode_state_read_once(inode) & I_DIRTY_TIME)) mark_inode_dirty_sync(inode); return file->f_op->fsync(file, start, end, datasync); } EXPORT_SYMBOL(vfs_fsync_range);

generic_write_sync内部调用vfs_fsync_range,在vfs_fsync_range定义上方的注释写得很清楚:helper to sync a range of data & metadata to disk,说明了"generic_write_sync负责将数据同步到磁盘"

结论

结论1: write内部调用generic_perform_write和generic_write_sync,其中 generic_perform_write负责将数据写入页缓存(Page Cache),generic_write_sync负责将数据同步到磁盘

继而得出结论2: 默认情况下(类似int fd = open("testfile", O_WRONLY|O_CREAT, 0644);write(fd, "hello", 5); 不含DAX和O_DIRECT标志),write先将数据写入到内核的缓冲区(准确来说是页缓存!),待时机成熟后再将缓冲区的数据刷新到磁盘

得出原因

通过进一步分析,使用close(1)导致fd=1的文件(stdout)被关闭,导致进程退出前自动刷新内核缓冲区时,无法将C语言提供的缓冲区中字符串的指针链接(不是拷贝,可以通过gdb的rwatch命令验证)到操作系统的缓冲区(因为找不到fd=1对应的缓冲区)然后刷新操作系统的缓冲区

那么可以在close(1)之前将C语言缓冲区的字符串写入到操作系统的缓冲区,可以使用write系统调用例如以下代码:

#include <stdio.h> #include <string.h> #include <unistd.h> int main() { const char* str1 = "hello printf"; const char* str2 = "hello fprintf"; const char* str3 = "hello fwrite"; write(1, str1, strlen(str1)); write(1, str2, strlen(str2)); write(1, str3, strlen(str3)); close(1); return 0; }

即使在关闭文件描述符fd=1后，之前使用write写入的数据也已经确实发送到显示器了

运行结果:

或者这样改:

#include <stdio.h> #include <string.h> #include <unistd.h> int main() { const char* str1 = "hello printf"; const char* str2 = "hello fprintf"; const char* str3 = "hello fwrite"; printf("%s",str1); // stdout -> 1 fprintf(stdout, "%s",str2); // stdout -> 1 fwrite(str3, strlen(str3), 1, stdout); // fwrite, stdout -> 1 fflush(stdout); close(1); return 0; }

运行结果:

查看printf、fprintf和fwrite写入的缓冲区

以含有fflush(stdout)的代码为例,先在这些地方下断点:

r命令执行

r

printf、fprintf和fwrite写入的缓冲区在stdout指向的FILE结构体中,gdb下可以这样查看stdout指向的FILE结构体:

p *stdout

c命令继续执行

c

再次查看stdout指向的FILE结构体,发现已经写入了一个字符串:

执行fprintf后,又写入了字符串:

执行fwrite后,又写入了字符串:

执行fflush(stdout),会将FILE结构体中的字符串的指针链接到操作系统的缓冲区中,而是否刷新到屏幕上是操作系统决定的,可以看C89文档的描述:fflush函数使得该流中任何未写入的数据被传送给主机环境,并写入文件

对于fflush内部的实现,可以看glibc在libio/iofflush.c的源码

结论: 用户刷新的本质,就是将数据通过fd=1和write系统调用写入到内核(上面提到的主机环境)中

(*注: 目前认为,只要将数据刷新到了内核,数据就到可以硬件了)

FILE结构体

由前面的分析可知: 文件操作绕不开FILE,FILE结构体里面含有打开文件的缓冲区字段和维护信息

那么打开n个文件就有n个文件描述符,也就有n个语言级别上的缓冲区

FILE结构体的源代码在glibc-2.42的libio/bits/types/FILE.h中给出了FILE的重定义:

#ifndef __FILE_defined #define __FILE_defined 1 struct _IO_FILE; /* The opaque type of streams. This is the definition used elsewhere. */ typedef struct _IO_FILE FILE; #endif

可以发现FILE是struct _IO_FILE的简写

在glibc-2.42的libio/bits/types/struct_FILE.h中,给出了结构体完整的定义:

/* The tag name of this struct is _IO_FILE to preserve historic C++ mangled names for functions taking FILE* arguments. That name should not be used in new code. */ struct _IO_FILE { int _flags; /* High-order word is _IO_MAGIC; rest is flags. */ /* The following pointers correspond to the C++ streambuf protocol. */ char *_IO_read_ptr; /* Current read pointer */ char *_IO_read_end; /* End of get area. */ char *_IO_read_base; /* Start of putback+get area. */ char *_IO_write_base; /* Start of put area. */ char *_IO_write_ptr; /* Current put pointer. */ char *_IO_write_end; /* End of put area. */ char *_IO_buf_base; /* Start of reserve area. */ char *_IO_buf_end; /* End of reserve area. */ /* The following fields are used to support backing up and undo. */ char *_IO_save_base; /* Pointer to start of non-current get area. */ char *_IO_backup_base; /* Pointer to first valid character of backup area */ char *_IO_save_end; /* Pointer to end of non-current get area. */ struct _IO_marker *_markers; struct _IO_FILE *_chain; int _fileno; int _flags2:24; /* Fallback buffer to use when malloc fails to allocate one. */ char _short_backupbuf[1]; __off_t _old_offset; /* This used to be _offset but it's too small. */ /* 1+column number of pbase(); 0 is unknown. */ unsigned short _cur_column; signed char _vtable_offset; char _shortbuf[1]; _IO_lock_t *_lock; #ifdef _IO_USE_OLD_IO_FILE };

1.可以看到里面的int _fileno就是文件描述符,原因如下:

因为IO相关函数与系统调用接口对应,并且库函数封装系统调用,所以本质上访问文件都是通过fd访问的,因此C库当中的FILE结构体内部必定封装了fd

2.FILE对象属于用户(因为是C语言标准规定的,而且编程语言属于用户层),里面的缓冲区是用户的缓冲区

3.每用C语言的fopen函数打开一个文件就要创建该文件的FILE对象

4.回顾exit和_exit

之前在OS23.【Linux】进程终止文章提到了exit和_exit,这里简单回顾:

1. exit是C语言退出函数,访问C语言提供的缓冲区是合情合理的,那么使用exit退出时会刷新C语言的缓冲区到内核

2. _exit是系统调用,作用也是退出,但和exit不一样的是: _exit处于底层,无法访问C语言提供的缓冲区,也就不会刷新C语言提供的缓冲区,_exit会close(1)然后结束进程

5.缓冲区刷新策略

C语言提供的缓冲区和操作系统内部的缓冲区的刷新策略有所不同,这里分开说

C语言缓冲区刷新所有方法

1.行缓冲: 直到见到\n才刷新,其余情况不刷新(例如显示器)

2.全缓冲: 缓冲区满了才刷新(例如普通文件的写入)

3.无缓冲: 直接刷新

如果是打印到显示器,行刷新或程序结束的时候刷新
如果是打印到文件,那么只有程序结束的时候才刷新

冲刷缓冲区的策略可能丢弃数据也可能将数据写入文件