当前位置: 技术问答>linux和unix
访问超过4G的文件,在Linux或Unix下该怎么做?
来源: 互联网 发布时间:2017-04-22
本文导语: 现在想读取一个大文件,可能是几个G或者几十G的文件,可否直接使用fopen打开,fread读取等平时我们用的方法? 在windows下是不可以的,不知道Unix、Linux下是否有限制,或者该怎么解决! 请给一小段代码实例,谢谢...
现在想读取一个大文件,可能是几个G或者几十G的文件,可否直接使用fopen打开,fread读取等平时我们用的方法?
在windows下是不可以的,不知道Unix、Linux下是否有限制,或者该怎么解决!
请给一小段代码实例,谢谢!
在windows下是不可以的,不知道Unix、Linux下是否有限制,或者该怎么解决!
请给一小段代码实例,谢谢!
|
http://bbs.chinaunix.net/thread-1612117-1-1.html
貌似编译的时候加个选项就可以了
貌似编译的时候加个选项就可以了
|
这个在linux下面叫large-file support (LFS),在源码开头加上
就可以了。
#define _FILE_OFFSET_BITS 64
就可以了。
|
AIX平台可以这样:
1、编译时增加宏定义_LARGE_FILES,如:xlC -D_LARGE_FILES t.cpp
2、使用fopen64代替fopen
3、程序编译为64位,如:xlC -q64 t.cpp
对于其他unix平台,lz可以尝试一下,编译为64位程序应该是通用的方式
1、编译时增加宏定义_LARGE_FILES,如:xlC -D_LARGE_FILES t.cpp
2、使用fopen64代替fopen
3、程序编译为64位,如:xlC -q64 t.cpp
对于其他unix平台,lz可以尝试一下,编译为64位程序应该是通用的方式
|
你用 open, write, read这些函数,你会发现里头的参数是read, write是size_t 或off_t (lseek);
所以你在编译的时候加入 -DLARGE_FILE - D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
即可!
应用层使用参数尽量用size_t, off_t。
所以你在编译的时候加入 -DLARGE_FILE - D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
即可!
应用层使用参数尽量用size_t, off_t。
|
在 windows 下可以用 CreateFile来打开文件 用 SetFilePointer来定位,用ReadFile来读取。这样多大的文件都可以读。
|
windows下用文件内存映射,参考《windows核心编程》17章----内存映射文件
linux下用mmap,参考《UNIX环境高级编程》14章----14.9 存储映射I/O
linux下用mmap,参考《UNIX环境高级编程》14章----14.9 存储映射I/O
|
#define _LARGEFILE_SOURCE
#define _LARGEFILE64_SOURCE
#define _FILE_OFFSET_BITS 64
#include
#include
#include
#include
int main(int argc, char *argv[])
{
int fd;
fd = open("a.txt", O_RDWR|O_CREAT|O_LARGEFILE, 0644);
if (fd==-1){
error("open");
exit(-1);
}
close(fd);
return 0;
}
如上代码定义上面那3个宏,然后再open的参数里加一个O_LARGEFILE,即可!
#define _LARGEFILE64_SOURCE
#define _FILE_OFFSET_BITS 64
#include
#include
#include
#include
int main(int argc, char *argv[])
{
int fd;
fd = open("a.txt", O_RDWR|O_CREAT|O_LARGEFILE, 0644);
if (fd==-1){
error("open");
exit(-1);
}
close(fd);
return 0;
}
如上代码定义上面那3个宏,然后再open的参数里加一个O_LARGEFILE,即可!
|
一个简单的问题竟然这么多人围观?
#man lseek64
不就啥都清楚了?
-----------------------------------------
NAME
lseek64 - reposition 64-bit read/write file offset
SYNOPSIS
#define _LARGEFILE64_SOURCE
#include
#include
off64_t lseek64(int fd, off64_t offset, int whence);
DESCRIPTION
The lseek(2) family of functions reposition the offset of the open file associated with the file descrip-
tor fd to offset bytes relative to the start, current position, or end of the file, when whence has the
value SEEK_SET, SEEK_CUR, or SEEK_END, respectively.
For more details, return value, and errors, see lseek(2).
Four interfaces are available: lseek(), lseek64(), llseek(), and the raw system call _llseek().
lseek
Prototype:
off_t lseek(int fd, off_t offset, int whence);
The library routine lseek() uses the type off_t. This is a 32-bit signed type on 32-bit architectures,
unless one compiles with
#define _FILE_OFFSET_BITS 64
in which case it is a 64-bit signed type.
lseek64
Prototype:
off64_t lseek64(int fd, off64_t offset, int whence);
The library routine lseek64() uses a 64-bit type even when off_t is a 32-bit type. Its prototype (and the
type off64_t) is available only when one compiles with
#define _LARGEFILE64_SOURCE
The function lseek64() is available since glibc 2.1, and is defined to be an alias for llseek().
。。。 。。。
#man lseek64
不就啥都清楚了?
-----------------------------------------
NAME
lseek64 - reposition 64-bit read/write file offset
SYNOPSIS
#define _LARGEFILE64_SOURCE
#include
#include
off64_t lseek64(int fd, off64_t offset, int whence);
DESCRIPTION
The lseek(2) family of functions reposition the offset of the open file associated with the file descrip-
tor fd to offset bytes relative to the start, current position, or end of the file, when whence has the
value SEEK_SET, SEEK_CUR, or SEEK_END, respectively.
For more details, return value, and errors, see lseek(2).
Four interfaces are available: lseek(), lseek64(), llseek(), and the raw system call _llseek().
lseek
Prototype:
off_t lseek(int fd, off_t offset, int whence);
The library routine lseek() uses the type off_t. This is a 32-bit signed type on 32-bit architectures,
unless one compiles with
#define _FILE_OFFSET_BITS 64
in which case it is a 64-bit signed type.
lseek64
Prototype:
off64_t lseek64(int fd, off64_t offset, int whence);
The library routine lseek64() uses a 64-bit type even when off_t is a 32-bit type. Its prototype (and the
type off64_t) is available only when one compiles with
#define _LARGEFILE64_SOURCE
The function lseek64() is available since glibc 2.1, and is defined to be an alias for llseek().
。。。 。。。
|
标准方法, 适用于POSIX兼容系统。
http://www.opengroup.org/onlinepubs/000095399/utilities/c99.html
getconf POSIX_V6_ILP32_OFFBIG_CFLAGS;
getconf POSIX_V6_ILP32_OFFBIG_LDFLAGS;
getconf POSIX_V6_ILP32_OFFBIG_LIBS;
http://www.opengroup.org/onlinepubs/000095399/utilities/c99.html
getconf POSIX_V6_ILP32_OFFBIG_CFLAGS;
getconf POSIX_V6_ILP32_OFFBIG_LDFLAGS;
getconf POSIX_V6_ILP32_OFFBIG_LIBS;