Lustre master sync v2 #5

In some cases such as kernel writeback, we shouldn't ignore the layout, otherwise, it could race with layout change undergoing. Intel-bug-id: LU-3160 Lustre-commit: 3141db609d95d379761e3b54899618b4037d38f6 Lustre-change: http://review.whamcloud.com/6154 Signed-off-by: Niu Yawei <yawei.niu@intel.com> Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com> Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> Reviewed-by: Fan Yong <fan.yong@intel.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com> [updated for upstream kernel submission] Signed-off-by: Peng Tao <tao.peng@emc.com>

Because ofd_get_info(KEY_FIEMAP) might be called from ptlrpc_server_handle_req_in(see the stack below), where env might not be initialized correctly(see LBUG below), so it refill refill in ofd_get_info. LutreError: 19182:0:(ofd_internal.h:518:ofd_info_init()) LBUG Pid: 19182, comm: ll_ost_io00_001 Call Trace: [<ffffffffa044e895>] libcfs_debug_dumpstack+0x55/0x80 [libcfs] [<ffffffffa044ee97>] lbug_with_loc+0x47/0xb0 [libcfs] [<ffffffffa0e03e62>] ofd_info_init+0x92/0x130 [ofd] [<ffffffffa0e05835>] ofd_get_info+0x2e5/0xa90 [ofd] [<ffffffff812805cd>] ? pointer+0x8d/0x830 [<ffffffffa029f7e5>] ? lprocfs_counter_add+0x125/0x182 [lvfs] [<ffffffffa078528a>] nrs_orr_range_fill_physical+0x18a/0x540 [ptlrpc] [<ffffffffa0762dd6>] ? __req_capsule_get+0x166/0x700 [ptlrpc] [<ffffffffa073e630>] ? lustre_swab_ost_body+0x0/0x10 [ptlrpc] [<ffffffffa07871d7>] nrs_orr_res_get+0x817/0xb80 [ptlrpc] [<ffffffffa077d306>] nrs_resource_get+0x56/0x110 [ptlrpc] [<ffffffffa077dccb>] nrs_resource_get_safe+0x8b/0x100 [ptlrpc] [<ffffffffa0780248>] ptlrpc_nrs_req_initialize+0x38/0x90 [ptlrpc] [<ffffffffa074cff0>] ptlrpc_main+0x1170/0x16f0 [ptlrpc] [<ffffffffa074be80>] ? ptlrpc_main+0x0/0x16f0 [ptlrpc] [<ffffffff8100c0ca>] child_rip+0xa/0x20 [<ffffffffa074be80>] ? ptlrpc_main+0x0/0x16f0 [ptlrpc] [<ffffffffa074be80>] ? ptlrpc_main+0x0/0x16f0 [ptlrpc] [<ffffffff8100c0c0>] ? child_rip+0x0/0x20 Intel-bug-id: LU-3239 Lustre-commit: 20b181a079e05359f5d54d6d361ff7f2e455d783 Lustre-change: http://review.whamcloud.com/6204 Signed-off-by: wang di <di.wang@intel.com> Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com> Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> Reviewed-by: Nikitas Angelinas <nikitas_angelinas@xyratex.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com> [updated for upstream kernel submission] Signed-off-by: Peng Tao <tao.peng@emc.com>

initialize oi.oi.oi_id which gcc 4.7.2 is afraid of being used later. Intel-bug-id: LU-3179 Lustre-commit: f2150a414ee84676a2e22809555b3b29898d8d12 Lustre-change: http://review.whamcloud.com/6064 Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com> Reviewed-by: wangdi <di.wang@intel.com> Reviewed-by: Bob Glossman <bob.glossman@intel.com> Reviewed-by: James Simmons <uja.ornl@gmail.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com> [updated for upstream kernel submission] Signed-off-by: Peng Tao <tao.peng@emc.com>

…etattr It should set echo seq before do echo getattr/setattr, otherwise echo_client will regard it as the object with MDT0 sequence. Intel-bug-id: LU-3267 Lustre-commit: 3e2d4d2d7477dd338a87de103027d37865285e44 Lustre-change: http://review.whamcloud.com/6263 Signed-off-by: wang di <di.wang@intel.com> Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> Reviewed-by: Lai Siyao <lai.siyao@intel.com> [picked client side change and updated for upstream kernel submission] Signed-off-by: Peng Tao <tao.peng@emc.com>

Quiet a number of overly noisy and unhelpful console error messages. Improve the format of other nearby errors. In the case of {lod,lov}_fix_desc_stripe_size(), this doesn't even need a console message unless it is actually changing some stripe size that is below the minimum. Typically it is only zero and is being bumped up to the default value. Intel-bug-id: LU-1095 Lustre-commit: da430ce6173149b4bafcad71ea1d650d54ea39c0 Lustre-change: http://review.whamcloud.com/6264 Signed-off-by: Andreas Dilger <andreas.dilger@intel.com> Reviewed-by: Bobi Jam <bobijam@gmail.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com> [updated for upstream kernel submission] Signed-off-by: Peng Tao <tao.peng@emc.com>

This seems to be causing multiple issues: LU-3274, LU-3277 Lustre-commit: fd2b619ca59739b1978fb3f09690526afa9d00dc [The original commit is folded in the large Lusre patch. So we don't have an exact commit to revert for kernel client -- Peng Tao] [updated for upstream kernel submission] Signed-off-by: Peng Tao <tao.peng@emc.com>

In ldlm_lock_decref_internal() when l_lvb_data is freed to reduce memory consumption, LDLM_FL_LVB_READY is not cleared, so later when the lock is reused lvb is not updated. But clearing LDLM_FL_LVB_READY forces layout refetch at each file access, so the better is to remove the optimization. The use case is after a restore in HSM. Intel-bug-id: LU-3280 Lustre-commit: e32ce066d231c8879aad52e518b508fa570af9ab Lustre-change: http://review.whamcloud.com/6268 Signed-off-by: JC Lafoucriere <jacques-charles.lafoucriere@cea.fr> Reviewed-by: Fan Yong <fan.yong@intel.com> Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com> [updated for upstream kernel submission] Signed-off-by: Peng Tao <tao.peng@emc.com>

…ance ptlrpcd_add_req() will wake_up other process, do not hold a spinlock before calling ptlrpcd_queue_work()->ptlrpcd_add_req(). If current process is allocating memory, memory shrinker could get to osc_lru_del(), don't call osc_lru_shrink() further since it could lead a long calling chain. Use static string OES_STRINGS in OSC_EXTENT_DUMP() to reduce stack footprint. Alloc crattr on heap for osc_build_rpc() to reduce stack footprint. Intel-bug-id: LU-3281 Lustre-commit: f7a81d4797933d179f9955bb0821779d3ac9a8fe Lustre-change: http://review.whamcloud.com/6270 Signed-off-by: Bobi Jam <bobijam.xu@intel.com> Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com> Reviewed-by: Keith Mannthey <keith.mannthey@intel.com> Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> [updated for upstream kernel submission] Signed-off-by: Peng Tao <tao.peng@emc.com>

This library is not used anymore and is replaced by local_storage.c. Patch removed last remnants of it. Intel-bug-id: LU-2886 Lustre-commit: 29e98f581ab68ecdfc1b923925eda2ec5b251f3a Lustre-change: http://review.whamcloud.com/6107 Signed-off-by: Mikhail Pershin <mike.pershin@intel.com> Reviewed-by: Fan Yong <fan.yong@intel.com> Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com> [updated for upstream kernel submission] Signed-off-by: Peng Tao <tao.peng@emc.com>

If lsm_lmm_verify_*() find an error in the lov_mds_md header structure, don't dump the full stripe information, since this can be totally bogus (e.g. if stripe_count == -1 or similar). Instead, just dump the header information for debugging. Intel-bug-id: LU-3271 Lustre-commit: ab28a55546afdd8871f5cd3dbcef86b97b68fd87 Lustre-change: http://review.whamcloud.com/6261 Signed-off-by: Andreas Dilger <andreas.dilger@intel.com> Reviewed-by: Bob Glossman <bob.glossman@intel.com> Reviewed-by: John Hammond <johnlockwoodhammond@gmail.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com> [updated for upstream kernel submission] Signed-off-by: Peng Tao <tao.peng@emc.com>

Local files last id are stored in root in files named seq-xxx-lastid while lastid for OST objects is stored in O/seq/LAST_ID special object with zero OID and handled by OSD. Patch reworks local files lastid to be stored in O/seq/LAST_ID too and using the same format. Intel-bug-id: LU-2886 Lustre-commit: c4677f0d97dff8d1084baa1792c0ad6af1000158 Lustre-change: http://review.whamcloud.com/6199 Signed-off-by: Mikhail Pershin <mike.pershin@intel.com> Signed-off-by: James Nunez <james.a.nunez@intel.com> Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com> Reviewed-by: Fan Yong <fan.yong@intel.com> [updated for upstream kernel submission] Signed-off-by: Peng Tao <tao.peng@emc.com>

Since logid still use id/seq format in the request, it will be swabbed by its own swab func, instead of using ostid swab, which might see logid as FID incorrectly. Intel-bug-id: LU-3302 Lustre-commit: 57d8d7f152b1a1cabc103193f859bd927b3fbe9c Lustre-change: http://review.whamcloud.com/6305 Signed-off-by: wang di <di.wang@intel.com> Reviewed-by: John Hammond <johnlockwoodhammond@gmail.com> Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> [updated for upstream kernel submission] Signed-off-by: Peng Tao <tao.peng@emc.com>

…tion Because old echo client still uses o_id/o_seq for objid, but new echo client will uses FID for the objid. Add OBD_CONNECT_FID for 2.4 echo client, so 2.4 OST will convert o_id/o_seq to FID if the request from old echo client. Add local flag OBD_FL_OSTID for o_flags to indicate OST does not support FID yet, then echo client will still send o_id/o_seq to OST. cleanup ost_validate_obdo Intel-bug-id: LU-3187 Lustre-commit: 00d9dff4fa51321b2185fcdc381ee8edb6ca62ce Lustre-change: http://review.whamcloud.com/6287 Signed-off-by: wang di <di.wang@intel.com> Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> Reviewed-by: Mike Pershin <mike.pershin@intel.com> [port client side change of the original Lustre commit] Signed-off-by: Peng Tao <tao.peng@emc.com>

In lprocfs_stats_seq_{start,next,show,stop}() encode the counter index (rather than the counter address) into *pos. Doing so simplifies these functions and fixes a bug in the case of per-CPU stats where no stats would be displayed at all if no events had yet occurred on CPU 0. Intel-bug-id: LU-2979 Lustre-commit: 6fe7c32906f67f43d815c884ed8a902bb2333992 Lustre-change: http://review.whamcloud.com/6328 Signed-off-by: John L. Hammond <john.hammond@intel.com> Reviewed-by: wangdi <di.wang@intel.com> Reviewed-by: Bobi Jam <bobijam@gmail.com> Reviewed-by: Emoly Liu <emoly.liu@intel.com> Reviewed-by: Keith Mannthey <keith.mannthey@intel.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com> [updated for upstream kernel submission] Signed-off-by: Peng Tao <tao.peng@emc.com>

In order to avoid compatibility issues with older ChangeLog consumers, change the new CL_LAYOUT record to use the value previously assigned to CL_IOCTL. The CL_IOCTL type was never used anywhere, and it didn't really make any sense as a ChangeLog record, since it could really mean anything at all. The changelog_show_cb() function is renamed to changelog_kkuc_cb(), since it is really about consuming the ChangeLog and passing it up to the kernel-user-coms interface. At some point we should consider to implement a DBUS-based ChangeLog interface as well. The changelog_kkuc_cb() "sanity check" on cr_type was removed, since there is no reason the client kernel needs to know every record type that is being passed to userspace. It is up to the client tool to determine what records that it can process. Intel-bug-id: LU-3279 Lustre-commit: dcd2b724b7c1e90b5f71625657dc6fb6107cfbe0 Lustre-change: http://review.whamcloud.com/6308 Signed-off-by: Andreas Dilger <andreas.dilger@intel.com> Reviewed-by: Bobi Jam <bobijam@gmail.com> Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com> Reviewed-by: Mike Pershin <mike.pershin@intel.com> Reviewed-by: Aurelien Degremont <aurelien.degremont@cea.fr> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com> [updated for upstream kernel submission] Signed-off-by: Peng Tao <tao.peng@emc.com>

Data corruption is possible if cp(coreutils) uses FIEMAP to obtain data holes in shared file, since there could be dirty cache on other clients which hasn't been flushed back. To ensure all the dirty on remote clients being flushed back on fiemap ioctl, we'd acquire ldlm lock on server side for fiemap, unless the local client (which invoke fiemap) has cached lock. Intel-bug-id: LU-3219 Lustre-commit: 58444c4e9bc58e192f0bc0c163a5d51d42ba4255 Xyratex-bug-id: MRP-1001 Lustre-change: http://review.whamcloud.com/6127 Signed-off-by: Artem Blagodarenko <artem_blagodarenko@xyratex.com> Signed-off-by: Niu Yawei <yawei.niu@intel.com> Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com> Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com> Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> [picked client side change and updated for upstream kernel submission] Signed-off-by: Peng Tao <tao.peng@emc.com>

Fix the CL_LAYOUT comment, since it is possible that layout swap could be used in cases where the file content is changed. I don't think there are any real world use cases for this, but that depends on what tools are being used on the filesystem. Also change the "LAYOUT" string for this ChangeLog record type to be "LYOUT" to match the convention of other strings to only be 5 characters long. Intel-bug-id: LU-3279 Lustre-commit: 930fcce5978c800cc09552d8bdfffdedc3fd535a Lustre-change: http://review.whamcloud.com/6338 Signed-off-by: Andreas Dilger <andreas.dilger@intel.com> Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com> [updated for upstream kernel submission] Signed-off-by: Peng Tao <tao.peng@emc.com>

…properly To make atime update properly between 1.8 client and 2.x server, this patch includes the following fixes: - if MDS_ATTR_xTIME is set without MDS_ATTR_xTIME_SET and the client does not have OBD_CONNECT_FULL20, convert it to LA_xTIME in mdt_setattr_unpack(). - set both MDS_ATTR_xTIME | MDS_ATTR_xTIME_SET for timestamps in ll_prepare_close(). This allows us to fix the server-side timestamp setting in the future. - remove attr_unpack() and convert the flags from MDS_ATTR_ to LA_* directly in mdt_attr_valid_xlate() instead. - improve sanityn.sh test_23(). Intel-bug-id: LU-3036 Lustre-commit: 565b6203761a09b28f6aee08c4dd4a5f6dbaf4f1 Lustre-change: http://review.whamcloud.com/6327 Signed-off-by: Liu Ying <emoly.liu@intel.com> Reviewed-by: Fan Yong <fan.yong@intel.com> Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> [updated for upstream kernel submission] Signed-off-by: Peng Tao <tao.peng@emc.com>

Always build 64bit ino internally except the client is mounted with "32bitapi" option, so client will always use 64bit ino internally. It will build 32bit ino, only if application requires 32 bit ino. Intel-bug-id: LU-3318 Lustre-commit: cf775525f5fa8c96d073809a6afc4af0643dda9a Lustre-change: http://review.whamcloud.com/6371 Signed-off-by: wang di <di.wang@intel.com> Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> Reviewed-by: Ned Bass <bass6@llnl.gov> Reviewed-by: Fan Yong <fan.yong@intel.com> [updated for upstream kernel submission] Signed-off-by: Peng Tao <tao.peng@emc.com>

…ectly In a7369bcd860af61a073cb2424139e3acccdcb203, I accidentally reversed the sense of the error check after the call to dt_statfs() in lprocfs_dt_rd_{blksize,{files,kbytes}{free,avail}. Unreverse the error checking. Intel-bug-id: LU-3300 Lustre-commit: 09b6f7b2f8832cca4dcf8378170438dcfd333beb Lustre-change: http://review.whamcloud.com/6385 Signed-off-by: John L. Hammond <john.hammond@intel.com> Reviewed-by: Robert Read <robert.read@intel.com> Reviewed-by: Emoly Liu <emoly.liu@intel.com> Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> [updated for upstream kernel submission] Signed-off-by: Peng Tao <tao.peng@emc.com>

Dump osc_extent waiting instead of ext in case error occurred. Intel-bug-id: LU-3359 Lustre-commit: 5a3610b711efff16cd14a098aa20f87a7055ba1a Lustre-change: http://review.whamcloud.com/6402 Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com> Reviewed-by: Lai Siyao <lai.siyao@intel.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com> [updated for upstream kernel submission] Signed-off-by: Peng Tao <tao.peng@emc.com>

This patch checks the number of bulk I/O RPC pages with PTLRPC_MAX_BRW_PAGES in ofd_preprw() to avoid LASSERT(iobuf->dr_npages < iobuf->dr_max_pages) occurring while larger I/O size is specified. The patch also fixes echo_client_prep_commit() to reuse the env context so as to avoid LASSERT(info->fti_exp == NULL) occurring while the bulk I/O size is larger than PTLRPC_MAX_BRW_SIZE. The patch also improves obdfilter-survey to handle the case while interoprating with old server. Intel-bug-id: LU-2598 Lustre-commit: fd7466439be3039cb27f723c57be73317b5e48ba Lustre-change: http://review.whamcloud.com/6394 Signed-off-by: Jian Yu <jian.yu@intel.com> Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com> Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> [updated for upstream kernel submission] Signed-off-by: Peng Tao <tao.peng@emc.com>

in lustre_set_wire_obdo, if "ocd->ocd_connect_flags" doesn't contain OBD_CONNECT_FID, use "ost_id" to check the type of the object instead of using ost_id.oi_fid only. Intel-bug-id: LU-3360 Lustre-commit: fed98fbd8f0c84c8e757bea96bcc7c251b4ffe30 Lustre-change: http://review.whamcloud.com/6426 Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com> Reviewed-by: wangdi <di.wang@intel.com> Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> [updated for upstream kernel submission] Signed-off-by: Peng Tao <tao.peng@emc.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Lustre master sync v2 #5

Lustre master sync v2 #5

Commits on May 30, 2013