19ec7b004SRick Macklem /*- 29ec7b004SRick Macklem * Copyright (c) 1989, 1993 39ec7b004SRick Macklem * The Regents of the University of California. All rights reserved. 49ec7b004SRick Macklem * 59ec7b004SRick Macklem * This code is derived from software contributed to Berkeley by 69ec7b004SRick Macklem * Rick Macklem at The University of Guelph. 79ec7b004SRick Macklem * 89ec7b004SRick Macklem * Redistribution and use in source and binary forms, with or without 99ec7b004SRick Macklem * modification, are permitted provided that the following conditions 109ec7b004SRick Macklem * are met: 119ec7b004SRick Macklem * 1. Redistributions of source code must retain the above copyright 129ec7b004SRick Macklem * notice, this list of conditions and the following disclaimer. 139ec7b004SRick Macklem * 2. Redistributions in binary form must reproduce the above copyright 149ec7b004SRick Macklem * notice, this list of conditions and the following disclaimer in the 159ec7b004SRick Macklem * documentation and/or other materials provided with the distribution. 16fbbd9655SWarner Losh * 3. Neither the name of the University nor the names of its contributors 179ec7b004SRick Macklem * may be used to endorse or promote products derived from this software 189ec7b004SRick Macklem * without specific prior written permission. 199ec7b004SRick Macklem * 209ec7b004SRick Macklem * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND 219ec7b004SRick Macklem * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 229ec7b004SRick Macklem * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 239ec7b004SRick Macklem * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE 249ec7b004SRick Macklem * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 259ec7b004SRick Macklem * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 269ec7b004SRick Macklem * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 279ec7b004SRick Macklem * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 289ec7b004SRick Macklem * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 299ec7b004SRick Macklem * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 309ec7b004SRick Macklem * SUCH DAMAGE. 319ec7b004SRick Macklem * 329ec7b004SRick Macklem */ 339ec7b004SRick Macklem 349ec7b004SRick Macklem #include <sys/cdefs.h> 359ec7b004SRick Macklem __FBSDID("$FreeBSD$"); 369ec7b004SRick Macklem 379ec7b004SRick Macklem /* 389ec7b004SRick Macklem * Here is the basic algorithm: 399ec7b004SRick Macklem * First, some design criteria I used: 409ec7b004SRick Macklem * - I think a false hit is more serious than a false miss 419ec7b004SRick Macklem * - A false hit for an RPC that has Op(s) that order via seqid# must be 429ec7b004SRick Macklem * avoided at all cost 439ec7b004SRick Macklem * - A valid hit will probably happen a long time after the original reply 449ec7b004SRick Macklem * and the TCP socket that the original request was received on will no 459ec7b004SRick Macklem * longer be active 469ec7b004SRick Macklem * (The long time delay implies to me that LRU is not appropriate.) 479ec7b004SRick Macklem * - The mechanism will satisfy the requirements of ordering Ops with seqid#s 489ec7b004SRick Macklem * in them as well as minimizing the risk of redoing retried non-idempotent 499ec7b004SRick Macklem * Ops. 509ec7b004SRick Macklem * Because it is biased towards avoiding false hits, multiple entries with 519ec7b004SRick Macklem * the same xid are to be expected, especially for the case of the entry 529ec7b004SRick Macklem * in the cache being related to a seqid# sequenced Op. 539ec7b004SRick Macklem * 549ec7b004SRick Macklem * The basic algorithm I'm about to code up: 559ec7b004SRick Macklem * - Null RPCs bypass the cache and are just done 569ec7b004SRick Macklem * For TCP 579ec7b004SRick Macklem * - key on <xid, NFS version> (as noted above, there can be several 589ec7b004SRick Macklem * entries with the same key) 599ec7b004SRick Macklem * When a request arrives: 609ec7b004SRick Macklem * For all that match key 619ec7b004SRick Macklem * - if RPC# != OR request_size != 629ec7b004SRick Macklem * - not a match with this one 639ec7b004SRick Macklem * - if NFSv4 and received on same TCP socket OR 649ec7b004SRick Macklem * received on a TCP connection created before the 659ec7b004SRick Macklem * entry was cached 669ec7b004SRick Macklem * - not a match with this one 679ec7b004SRick Macklem * (V2,3 clients might retry on same TCP socket) 689ec7b004SRick Macklem * - calculate checksum on first N bytes of NFS XDR 699ec7b004SRick Macklem * - if checksum != 709ec7b004SRick Macklem * - not a match for this one 719ec7b004SRick Macklem * If any of the remaining ones that match has a 729ec7b004SRick Macklem * seqid_refcnt > 0 739ec7b004SRick Macklem * - not a match (go do RPC, using new cache entry) 749ec7b004SRick Macklem * If one match left 759ec7b004SRick Macklem * - a hit (reply from cache) 769ec7b004SRick Macklem * else 779ec7b004SRick Macklem * - miss (go do RPC, using new cache entry) 789ec7b004SRick Macklem * 799ec7b004SRick Macklem * During processing of NFSv4 request: 809ec7b004SRick Macklem * - set a flag when a non-idempotent Op is processed 819ec7b004SRick Macklem * - when an Op that uses a seqid# (Open,...) is processed 829ec7b004SRick Macklem * - if same seqid# as referenced entry in cache 839ec7b004SRick Macklem * - free new cache entry 849ec7b004SRick Macklem * - reply from referenced cache entry 859ec7b004SRick Macklem * else if next seqid# in order 869ec7b004SRick Macklem * - free referenced cache entry 879ec7b004SRick Macklem * - increment seqid_refcnt on new cache entry 889ec7b004SRick Macklem * - set pointer from Openowner/Lockowner to 899ec7b004SRick Macklem * new cache entry (aka reference it) 909ec7b004SRick Macklem * else if first seqid# in sequence 919ec7b004SRick Macklem * - increment seqid_refcnt on new cache entry 929ec7b004SRick Macklem * - set pointer from Openowner/Lockowner to 939ec7b004SRick Macklem * new cache entry (aka reference it) 949ec7b004SRick Macklem * 959ec7b004SRick Macklem * At end of RPC processing: 969ec7b004SRick Macklem * - if seqid_refcnt > 0 OR flagged non-idempotent on new 979ec7b004SRick Macklem * cache entry 989ec7b004SRick Macklem * - save reply in cache entry 999ec7b004SRick Macklem * - calculate checksum on first N bytes of NFS XDR 1009ec7b004SRick Macklem * request 1019ec7b004SRick Macklem * - note op and length of XDR request (in bytes) 1029ec7b004SRick Macklem * - timestamp it 1039ec7b004SRick Macklem * else 1049ec7b004SRick Macklem * - free new cache entry 1059ec7b004SRick Macklem * - Send reply (noting info for socket activity check, below) 1069ec7b004SRick Macklem * 1079ec7b004SRick Macklem * For cache entries saved above: 1089ec7b004SRick Macklem * - if saved since seqid_refcnt was > 0 1099ec7b004SRick Macklem * - free when seqid_refcnt decrements to 0 1109ec7b004SRick Macklem * (when next one in sequence is processed above, or 1119ec7b004SRick Macklem * when Openowner/Lockowner is discarded) 1129ec7b004SRick Macklem * else { non-idempotent Op(s) } 1139ec7b004SRick Macklem * - free when 1149ec7b004SRick Macklem * - some further activity observed on same 1159ec7b004SRick Macklem * socket 1169ec7b004SRick Macklem * (I'm not yet sure how I'm going to do 1179ec7b004SRick Macklem * this. Maybe look at the TCP connection 1189ec7b004SRick Macklem * to see if the send_tcp_sequence# is well 1199ec7b004SRick Macklem * past sent reply OR K additional RPCs 1209ec7b004SRick Macklem * replied on same socket OR?) 1219ec7b004SRick Macklem * OR 1229ec7b004SRick Macklem * - when very old (hours, days, weeks?) 1239ec7b004SRick Macklem * 1249ec7b004SRick Macklem * For UDP (v2, 3 only), pretty much the old way: 1259ec7b004SRick Macklem * - key on <xid, NFS version, RPC#, Client host ip#> 1269ec7b004SRick Macklem * (at most one entry for each key) 1279ec7b004SRick Macklem * 1289ec7b004SRick Macklem * When a Request arrives: 1299ec7b004SRick Macklem * - if a match with entry via key 1309ec7b004SRick Macklem * - if RPC marked In_progress 1319ec7b004SRick Macklem * - discard request (don't send reply) 1329ec7b004SRick Macklem * else 1339ec7b004SRick Macklem * - reply from cache 1349ec7b004SRick Macklem * - timestamp cache entry 1359ec7b004SRick Macklem * else 1369ec7b004SRick Macklem * - add entry to cache, marked In_progress 1379ec7b004SRick Macklem * - do RPC 1389ec7b004SRick Macklem * - when RPC done 1399ec7b004SRick Macklem * - if RPC# non-idempotent 1409ec7b004SRick Macklem * - mark entry Done (not In_progress) 1419ec7b004SRick Macklem * - save reply 1429ec7b004SRick Macklem * - timestamp cache entry 1439ec7b004SRick Macklem * else 1449ec7b004SRick Macklem * - free cache entry 1459ec7b004SRick Macklem * - send reply 1469ec7b004SRick Macklem * 1479ec7b004SRick Macklem * Later, entries with saved replies are free'd a short time (few minutes) 1489ec7b004SRick Macklem * after reply sent (timestamp). 1499ec7b004SRick Macklem * Reference: Chet Juszczak, "Improving the Performance and Correctness 1509ec7b004SRick Macklem * of an NFS Server", in Proc. Winter 1989 USENIX Conference, 1519ec7b004SRick Macklem * pages 53-63. San Diego, February 1989. 1529ec7b004SRick Macklem * for the UDP case. 1539ec7b004SRick Macklem * nfsrc_floodlevel is set to the allowable upper limit for saved replies 1549ec7b004SRick Macklem * for TCP. For V3, a reply won't be saved when the flood level is 1559ec7b004SRick Macklem * hit. For V4, the non-idempotent Op will return NFSERR_RESOURCE in 1569ec7b004SRick Macklem * that case. This level should be set high enough that this almost 1579ec7b004SRick Macklem * never happens. 1589ec7b004SRick Macklem */ 1599ec7b004SRick Macklem #ifndef APPLEKEXT 1609ec7b004SRick Macklem #include <fs/nfs/nfsport.h> 1619ec7b004SRick Macklem 1621b819cf2SRick Macklem extern struct nfsstatsv1 nfsstatsv1; 16393c5875bSRick Macklem extern struct mtx nfsrc_udpmtx; 16493c5875bSRick Macklem extern struct nfsrchash_bucket nfsrchash_table[NFSRVCACHE_HASHSIZE]; 165d473bac7SAlexander Motin extern struct nfsrchash_bucket nfsrcahash_table[NFSRVCACHE_HASHSIZE]; 1669ec7b004SRick Macklem int nfsrc_floodlevel = NFSRVCACHE_FLOODLEVEL, nfsrc_tcpsavedreplies = 0; 1679ec7b004SRick Macklem #endif /* !APPLEKEXT */ 1689ec7b004SRick Macklem 16993c5875bSRick Macklem SYSCTL_DECL(_vfs_nfsd); 17093c5875bSRick Macklem 17193c5875bSRick Macklem static u_int nfsrc_tcphighwater = 0; 17293c5875bSRick Macklem static int 17393c5875bSRick Macklem sysctl_tcphighwater(SYSCTL_HANDLER_ARGS) 17493c5875bSRick Macklem { 17593c5875bSRick Macklem int error, newhighwater; 17693c5875bSRick Macklem 17793c5875bSRick Macklem newhighwater = nfsrc_tcphighwater; 17893c5875bSRick Macklem error = sysctl_handle_int(oidp, &newhighwater, 0, req); 17993c5875bSRick Macklem if (error != 0 || req->newptr == NULL) 18093c5875bSRick Macklem return (error); 18193c5875bSRick Macklem if (newhighwater < 0) 18293c5875bSRick Macklem return (EINVAL); 18393c5875bSRick Macklem if (newhighwater >= nfsrc_floodlevel) 18493c5875bSRick Macklem nfsrc_floodlevel = newhighwater + newhighwater / 5; 18593c5875bSRick Macklem nfsrc_tcphighwater = newhighwater; 18693c5875bSRick Macklem return (0); 18793c5875bSRick Macklem } 18893c5875bSRick Macklem SYSCTL_PROC(_vfs_nfsd, OID_AUTO, tcphighwater, CTLTYPE_UINT | CTLFLAG_RW, 0, 18993c5875bSRick Macklem sizeof(nfsrc_tcphighwater), sysctl_tcphighwater, "IU", 19093c5875bSRick Macklem "High water mark for TCP cache entries"); 19193c5875bSRick Macklem 19293c5875bSRick Macklem static u_int nfsrc_udphighwater = NFSRVCACHE_UDPHIGHWATER; 19393c5875bSRick Macklem SYSCTL_UINT(_vfs_nfsd, OID_AUTO, udphighwater, CTLFLAG_RW, 19493c5875bSRick Macklem &nfsrc_udphighwater, 0, 19593c5875bSRick Macklem "High water mark for UDP cache entries"); 19693c5875bSRick Macklem static u_int nfsrc_tcptimeout = NFSRVCACHE_TCPTIMEOUT; 19793c5875bSRick Macklem SYSCTL_UINT(_vfs_nfsd, OID_AUTO, tcpcachetimeo, CTLFLAG_RW, 19893c5875bSRick Macklem &nfsrc_tcptimeout, 0, 19993c5875bSRick Macklem "Timeout for TCP entries in the DRC"); 20093c5875bSRick Macklem static u_int nfsrc_tcpnonidempotent = 1; 20193c5875bSRick Macklem SYSCTL_UINT(_vfs_nfsd, OID_AUTO, cachetcp, CTLFLAG_RW, 20293c5875bSRick Macklem &nfsrc_tcpnonidempotent, 0, 20393c5875bSRick Macklem "Enable the DRC for NFS over TCP"); 20493c5875bSRick Macklem 20593c5875bSRick Macklem static int nfsrc_udpcachesize = 0; 2069ec7b004SRick Macklem static TAILQ_HEAD(, nfsrvcache) nfsrvudplru; 20793c5875bSRick Macklem static struct nfsrvhashhead nfsrvudphashtbl[NFSRVCACHE_HASHSIZE]; 20893c5875bSRick Macklem 2099ec7b004SRick Macklem /* 2109ec7b004SRick Macklem * and the reverse mapping from generic to Version 2 procedure numbers 2119ec7b004SRick Macklem */ 2129ec7b004SRick Macklem static int newnfsv2_procid[NFS_V3NPROCS] = { 2139ec7b004SRick Macklem NFSV2PROC_NULL, 2149ec7b004SRick Macklem NFSV2PROC_GETATTR, 2159ec7b004SRick Macklem NFSV2PROC_SETATTR, 2169ec7b004SRick Macklem NFSV2PROC_LOOKUP, 2179ec7b004SRick Macklem NFSV2PROC_NOOP, 2189ec7b004SRick Macklem NFSV2PROC_READLINK, 2199ec7b004SRick Macklem NFSV2PROC_READ, 2209ec7b004SRick Macklem NFSV2PROC_WRITE, 2219ec7b004SRick Macklem NFSV2PROC_CREATE, 2229ec7b004SRick Macklem NFSV2PROC_MKDIR, 2239ec7b004SRick Macklem NFSV2PROC_SYMLINK, 2249ec7b004SRick Macklem NFSV2PROC_CREATE, 2259ec7b004SRick Macklem NFSV2PROC_REMOVE, 2269ec7b004SRick Macklem NFSV2PROC_RMDIR, 2279ec7b004SRick Macklem NFSV2PROC_RENAME, 2289ec7b004SRick Macklem NFSV2PROC_LINK, 2299ec7b004SRick Macklem NFSV2PROC_READDIR, 2309ec7b004SRick Macklem NFSV2PROC_NOOP, 2319ec7b004SRick Macklem NFSV2PROC_STATFS, 2329ec7b004SRick Macklem NFSV2PROC_NOOP, 2339ec7b004SRick Macklem NFSV2PROC_NOOP, 2349ec7b004SRick Macklem NFSV2PROC_NOOP, 2359ec7b004SRick Macklem }; 2369ec7b004SRick Macklem 23793c5875bSRick Macklem #define nfsrc_hash(xid) (((xid) + ((xid) >> 24)) % NFSRVCACHE_HASHSIZE) 2389ec7b004SRick Macklem #define NFSRCUDPHASH(xid) \ 23993c5875bSRick Macklem (&nfsrvudphashtbl[nfsrc_hash(xid)]) 2409ec7b004SRick Macklem #define NFSRCHASH(xid) \ 24193c5875bSRick Macklem (&nfsrchash_table[nfsrc_hash(xid)].tbl) 242d473bac7SAlexander Motin #define NFSRCAHASH(xid) (&nfsrcahash_table[nfsrc_hash(xid)]) 2439ec7b004SRick Macklem #define TRUE 1 2449ec7b004SRick Macklem #define FALSE 0 2459ec7b004SRick Macklem #define NFSRVCACHE_CHECKLEN 100 2469ec7b004SRick Macklem 2479ec7b004SRick Macklem /* True iff the rpc reply is an nfs status ONLY! */ 2489ec7b004SRick Macklem static int nfsv2_repstat[NFS_V3NPROCS] = { 2499ec7b004SRick Macklem FALSE, 2509ec7b004SRick Macklem FALSE, 2519ec7b004SRick Macklem FALSE, 2529ec7b004SRick Macklem FALSE, 2539ec7b004SRick Macklem FALSE, 2549ec7b004SRick Macklem FALSE, 2559ec7b004SRick Macklem FALSE, 2569ec7b004SRick Macklem FALSE, 2579ec7b004SRick Macklem FALSE, 2589ec7b004SRick Macklem FALSE, 2599ec7b004SRick Macklem TRUE, 2609ec7b004SRick Macklem TRUE, 2619ec7b004SRick Macklem TRUE, 2629ec7b004SRick Macklem TRUE, 2639ec7b004SRick Macklem FALSE, 2649ec7b004SRick Macklem TRUE, 2659ec7b004SRick Macklem FALSE, 2669ec7b004SRick Macklem FALSE, 2679ec7b004SRick Macklem FALSE, 2689ec7b004SRick Macklem FALSE, 2699ec7b004SRick Macklem FALSE, 2709ec7b004SRick Macklem FALSE, 2719ec7b004SRick Macklem }; 2729ec7b004SRick Macklem 2739ec7b004SRick Macklem /* 2749ec7b004SRick Macklem * Will NFS want to work over IPv6 someday? 2759ec7b004SRick Macklem */ 2769ec7b004SRick Macklem #define NETFAMILY(rp) \ 2779ec7b004SRick Macklem (((rp)->rc_flag & RC_INETIPV6) ? AF_INET6 : AF_INET) 2789ec7b004SRick Macklem 2799ec7b004SRick Macklem /* local functions */ 2809ec7b004SRick Macklem static int nfsrc_getudp(struct nfsrv_descript *nd, struct nfsrvcache *newrp); 2819ec7b004SRick Macklem static int nfsrc_gettcp(struct nfsrv_descript *nd, struct nfsrvcache *newrp); 2829ec7b004SRick Macklem static void nfsrc_lock(struct nfsrvcache *rp); 2839ec7b004SRick Macklem static void nfsrc_unlock(struct nfsrvcache *rp); 2849ec7b004SRick Macklem static void nfsrc_wanted(struct nfsrvcache *rp); 2859ec7b004SRick Macklem static void nfsrc_freecache(struct nfsrvcache *rp); 2869ec7b004SRick Macklem static int nfsrc_getlenandcksum(mbuf_t m1, u_int16_t *cksum); 2879ec7b004SRick Macklem static void nfsrc_marksametcpconn(u_int64_t); 2889ec7b004SRick Macklem 2899ec7b004SRick Macklem /* 29093c5875bSRick Macklem * Return the correct mutex for this cache entry. 29193c5875bSRick Macklem */ 29293c5875bSRick Macklem static __inline struct mtx * 29393c5875bSRick Macklem nfsrc_cachemutex(struct nfsrvcache *rp) 29493c5875bSRick Macklem { 29593c5875bSRick Macklem 29693c5875bSRick Macklem if ((rp->rc_flag & RC_UDP) != 0) 29793c5875bSRick Macklem return (&nfsrc_udpmtx); 29893c5875bSRick Macklem return (&nfsrchash_table[nfsrc_hash(rp->rc_xid)].mtx); 29993c5875bSRick Macklem } 30093c5875bSRick Macklem 30193c5875bSRick Macklem /* 3029ec7b004SRick Macklem * Initialize the server request cache list 3039ec7b004SRick Macklem */ 3049ec7b004SRick Macklem APPLESTATIC void 3059ec7b004SRick Macklem nfsrvd_initcache(void) 3069ec7b004SRick Macklem { 3079ec7b004SRick Macklem int i; 3089ec7b004SRick Macklem static int inited = 0; 3099ec7b004SRick Macklem 3109ec7b004SRick Macklem if (inited) 3119ec7b004SRick Macklem return; 3129ec7b004SRick Macklem inited = 1; 3139ec7b004SRick Macklem for (i = 0; i < NFSRVCACHE_HASHSIZE; i++) { 3149ec7b004SRick Macklem LIST_INIT(&nfsrvudphashtbl[i]); 31593c5875bSRick Macklem LIST_INIT(&nfsrchash_table[i].tbl); 316d473bac7SAlexander Motin LIST_INIT(&nfsrcahash_table[i].tbl); 3179ec7b004SRick Macklem } 3189ec7b004SRick Macklem TAILQ_INIT(&nfsrvudplru); 3199ec7b004SRick Macklem nfsrc_tcpsavedreplies = 0; 3209ec7b004SRick Macklem nfsrc_udpcachesize = 0; 3211b819cf2SRick Macklem nfsstatsv1.srvcache_tcppeak = 0; 3221b819cf2SRick Macklem nfsstatsv1.srvcache_size = 0; 3239ec7b004SRick Macklem } 3249ec7b004SRick Macklem 3259ec7b004SRick Macklem /* 3269ec7b004SRick Macklem * Get a cache entry for this request. Basically just malloc a new one 3279ec7b004SRick Macklem * and then call nfsrc_getudp() or nfsrc_gettcp() to do the rest. 3289ec7b004SRick Macklem */ 3299ec7b004SRick Macklem APPLESTATIC int 330d473bac7SAlexander Motin nfsrvd_getcache(struct nfsrv_descript *nd) 3319ec7b004SRick Macklem { 3329ec7b004SRick Macklem struct nfsrvcache *newrp; 3339ec7b004SRick Macklem int ret; 3349ec7b004SRick Macklem 3359ec7b004SRick Macklem if (nd->nd_procnum == NFSPROC_NULL) 3369ec7b004SRick Macklem panic("nfsd cache null"); 3379ec7b004SRick Macklem MALLOC(newrp, struct nfsrvcache *, sizeof (struct nfsrvcache), 3389ec7b004SRick Macklem M_NFSRVCACHE, M_WAITOK); 3399ec7b004SRick Macklem NFSBZERO((caddr_t)newrp, sizeof (struct nfsrvcache)); 3409ec7b004SRick Macklem if (nd->nd_flag & ND_NFSV4) 3419ec7b004SRick Macklem newrp->rc_flag = RC_NFSV4; 3429ec7b004SRick Macklem else if (nd->nd_flag & ND_NFSV3) 3439ec7b004SRick Macklem newrp->rc_flag = RC_NFSV3; 3449ec7b004SRick Macklem else 3459ec7b004SRick Macklem newrp->rc_flag = RC_NFSV2; 3469ec7b004SRick Macklem newrp->rc_xid = nd->nd_retxid; 3479ec7b004SRick Macklem newrp->rc_proc = nd->nd_procnum; 3489ec7b004SRick Macklem newrp->rc_sockref = nd->nd_sockref; 3499ec7b004SRick Macklem newrp->rc_cachetime = nd->nd_tcpconntime; 3509ec7b004SRick Macklem if (nd->nd_flag & ND_SAMETCPCONN) 3519ec7b004SRick Macklem newrp->rc_flag |= RC_SAMETCPCONN; 3529ec7b004SRick Macklem if (nd->nd_nam2 != NULL) { 3539ec7b004SRick Macklem newrp->rc_flag |= RC_UDP; 3549ec7b004SRick Macklem ret = nfsrc_getudp(nd, newrp); 3559ec7b004SRick Macklem } else { 3569ec7b004SRick Macklem ret = nfsrc_gettcp(nd, newrp); 3579ec7b004SRick Macklem } 358a9285ae5SZack Kirsch NFSEXITCODE2(0, nd); 3599ec7b004SRick Macklem return (ret); 3609ec7b004SRick Macklem } 3619ec7b004SRick Macklem 3629ec7b004SRick Macklem /* 3639ec7b004SRick Macklem * For UDP (v2, v3): 3649ec7b004SRick Macklem * - key on <xid, NFS version, RPC#, Client host ip#> 3659ec7b004SRick Macklem * (at most one entry for each key) 3669ec7b004SRick Macklem */ 3679ec7b004SRick Macklem static int 3689ec7b004SRick Macklem nfsrc_getudp(struct nfsrv_descript *nd, struct nfsrvcache *newrp) 3699ec7b004SRick Macklem { 3709ec7b004SRick Macklem struct nfsrvcache *rp; 3719ec7b004SRick Macklem struct sockaddr_in *saddr; 3729ec7b004SRick Macklem struct sockaddr_in6 *saddr6; 3739ec7b004SRick Macklem struct nfsrvhashhead *hp; 3749ec7b004SRick Macklem int ret = 0; 37593c5875bSRick Macklem struct mtx *mutex; 3769ec7b004SRick Macklem 37793c5875bSRick Macklem mutex = nfsrc_cachemutex(newrp); 3789ec7b004SRick Macklem hp = NFSRCUDPHASH(newrp->rc_xid); 3799ec7b004SRick Macklem loop: 38093c5875bSRick Macklem mtx_lock(mutex); 3819ec7b004SRick Macklem LIST_FOREACH(rp, hp, rc_hash) { 3829ec7b004SRick Macklem if (newrp->rc_xid == rp->rc_xid && 3839ec7b004SRick Macklem newrp->rc_proc == rp->rc_proc && 3849ec7b004SRick Macklem (newrp->rc_flag & rp->rc_flag & RC_NFSVERS) && 3859ec7b004SRick Macklem nfsaddr_match(NETFAMILY(rp), &rp->rc_haddr, nd->nd_nam)) { 3869ec7b004SRick Macklem if ((rp->rc_flag & RC_LOCKED) != 0) { 3879ec7b004SRick Macklem rp->rc_flag |= RC_WANTED; 38893c5875bSRick Macklem (void)mtx_sleep(rp, mutex, (PZERO - 1) | PDROP, 38993c5875bSRick Macklem "nfsrc", 10 * hz); 3909ec7b004SRick Macklem goto loop; 3919ec7b004SRick Macklem } 3929ec7b004SRick Macklem if (rp->rc_flag == 0) 3939ec7b004SRick Macklem panic("nfs udp cache0"); 3949ec7b004SRick Macklem rp->rc_flag |= RC_LOCKED; 3959ec7b004SRick Macklem TAILQ_REMOVE(&nfsrvudplru, rp, rc_lru); 3969ec7b004SRick Macklem TAILQ_INSERT_TAIL(&nfsrvudplru, rp, rc_lru); 3979ec7b004SRick Macklem if (rp->rc_flag & RC_INPROG) { 3981b819cf2SRick Macklem nfsstatsv1.srvcache_inproghits++; 39993c5875bSRick Macklem mtx_unlock(mutex); 4009ec7b004SRick Macklem ret = RC_DROPIT; 4019ec7b004SRick Macklem } else if (rp->rc_flag & RC_REPSTATUS) { 4029ec7b004SRick Macklem /* 4039ec7b004SRick Macklem * V2 only. 4049ec7b004SRick Macklem */ 4051b819cf2SRick Macklem nfsstatsv1.srvcache_nonidemdonehits++; 40693c5875bSRick Macklem mtx_unlock(mutex); 4079ec7b004SRick Macklem nfsrvd_rephead(nd); 4089ec7b004SRick Macklem *(nd->nd_errp) = rp->rc_status; 4099ec7b004SRick Macklem ret = RC_REPLY; 4109ec7b004SRick Macklem rp->rc_timestamp = NFSD_MONOSEC + 4119ec7b004SRick Macklem NFSRVCACHE_UDPTIMEOUT; 4129ec7b004SRick Macklem } else if (rp->rc_flag & RC_REPMBUF) { 4131b819cf2SRick Macklem nfsstatsv1.srvcache_nonidemdonehits++; 41493c5875bSRick Macklem mtx_unlock(mutex); 4159ec7b004SRick Macklem nd->nd_mreq = m_copym(rp->rc_reply, 0, 416eb1b1807SGleb Smirnoff M_COPYALL, M_WAITOK); 4179ec7b004SRick Macklem ret = RC_REPLY; 4189ec7b004SRick Macklem rp->rc_timestamp = NFSD_MONOSEC + 4199ec7b004SRick Macklem NFSRVCACHE_UDPTIMEOUT; 4209ec7b004SRick Macklem } else { 4219ec7b004SRick Macklem panic("nfs udp cache1"); 4229ec7b004SRick Macklem } 4239ec7b004SRick Macklem nfsrc_unlock(rp); 4249ec7b004SRick Macklem free((caddr_t)newrp, M_NFSRVCACHE); 425a9285ae5SZack Kirsch goto out; 4269ec7b004SRick Macklem } 4279ec7b004SRick Macklem } 4281b819cf2SRick Macklem nfsstatsv1.srvcache_misses++; 4291b819cf2SRick Macklem atomic_add_int(&nfsstatsv1.srvcache_size, 1); 4309ec7b004SRick Macklem nfsrc_udpcachesize++; 4319ec7b004SRick Macklem 4329ec7b004SRick Macklem newrp->rc_flag |= RC_INPROG; 4339ec7b004SRick Macklem saddr = NFSSOCKADDR(nd->nd_nam, struct sockaddr_in *); 4349ec7b004SRick Macklem if (saddr->sin_family == AF_INET) 4359ec7b004SRick Macklem newrp->rc_inet = saddr->sin_addr.s_addr; 4369ec7b004SRick Macklem else if (saddr->sin_family == AF_INET6) { 4379ec7b004SRick Macklem saddr6 = (struct sockaddr_in6 *)saddr; 4389ec7b004SRick Macklem NFSBCOPY((caddr_t)&saddr6->sin6_addr, (caddr_t)&newrp->rc_inet6, 4399ec7b004SRick Macklem sizeof (struct in6_addr)); 440d5ad6625SRick Macklem newrp->rc_flag |= RC_INETIPV6; 4419ec7b004SRick Macklem } 4429ec7b004SRick Macklem LIST_INSERT_HEAD(hp, newrp, rc_hash); 4439ec7b004SRick Macklem TAILQ_INSERT_TAIL(&nfsrvudplru, newrp, rc_lru); 44493c5875bSRick Macklem mtx_unlock(mutex); 4459ec7b004SRick Macklem nd->nd_rp = newrp; 446a9285ae5SZack Kirsch ret = RC_DOIT; 447a9285ae5SZack Kirsch 448a9285ae5SZack Kirsch out: 449a9285ae5SZack Kirsch NFSEXITCODE2(0, nd); 450a9285ae5SZack Kirsch return (ret); 4519ec7b004SRick Macklem } 4529ec7b004SRick Macklem 4539ec7b004SRick Macklem /* 4549ec7b004SRick Macklem * Update a request cache entry after the rpc has been done 4559ec7b004SRick Macklem */ 4569ec7b004SRick Macklem APPLESTATIC struct nfsrvcache * 457d473bac7SAlexander Motin nfsrvd_updatecache(struct nfsrv_descript *nd) 4589ec7b004SRick Macklem { 4599ec7b004SRick Macklem struct nfsrvcache *rp; 4609ec7b004SRick Macklem struct nfsrvcache *retrp = NULL; 4614e22c98aSRick Macklem mbuf_t m; 46293c5875bSRick Macklem struct mtx *mutex; 4639ec7b004SRick Macklem 4649ec7b004SRick Macklem rp = nd->nd_rp; 4659ec7b004SRick Macklem if (!rp) 4669ec7b004SRick Macklem panic("nfsrvd_updatecache null rp"); 4679ec7b004SRick Macklem nd->nd_rp = NULL; 46893c5875bSRick Macklem mutex = nfsrc_cachemutex(rp); 46993c5875bSRick Macklem mtx_lock(mutex); 4709ec7b004SRick Macklem nfsrc_lock(rp); 4719ec7b004SRick Macklem if (!(rp->rc_flag & RC_INPROG)) 4729ec7b004SRick Macklem panic("nfsrvd_updatecache not inprog"); 4739ec7b004SRick Macklem rp->rc_flag &= ~RC_INPROG; 4749ec7b004SRick Macklem if (rp->rc_flag & RC_UDP) { 4759ec7b004SRick Macklem TAILQ_REMOVE(&nfsrvudplru, rp, rc_lru); 4769ec7b004SRick Macklem TAILQ_INSERT_TAIL(&nfsrvudplru, rp, rc_lru); 4779ec7b004SRick Macklem } 4789ec7b004SRick Macklem 4799ec7b004SRick Macklem /* 4809ec7b004SRick Macklem * Reply from cache is a special case returned by nfsrv_checkseqid(). 4819ec7b004SRick Macklem */ 4829ec7b004SRick Macklem if (nd->nd_repstat == NFSERR_REPLYFROMCACHE) { 4831b819cf2SRick Macklem nfsstatsv1.srvcache_nonidemdonehits++; 48493c5875bSRick Macklem mtx_unlock(mutex); 4859ec7b004SRick Macklem nd->nd_repstat = 0; 4869ec7b004SRick Macklem if (nd->nd_mreq) 4879ec7b004SRick Macklem mbuf_freem(nd->nd_mreq); 4889ec7b004SRick Macklem if (!(rp->rc_flag & RC_REPMBUF)) 4899ec7b004SRick Macklem panic("reply from cache"); 4909ec7b004SRick Macklem nd->nd_mreq = m_copym(rp->rc_reply, 0, 491eb1b1807SGleb Smirnoff M_COPYALL, M_WAITOK); 49293c5875bSRick Macklem rp->rc_timestamp = NFSD_MONOSEC + nfsrc_tcptimeout; 4939ec7b004SRick Macklem nfsrc_unlock(rp); 494a9285ae5SZack Kirsch goto out; 4959ec7b004SRick Macklem } 4969ec7b004SRick Macklem 4979ec7b004SRick Macklem /* 4989ec7b004SRick Macklem * If rc_refcnt > 0, save it 4999ec7b004SRick Macklem * For UDP, save it if ND_SAVEREPLY is set 5009ec7b004SRick Macklem * For TCP, save it if ND_SAVEREPLY and nfsrc_tcpnonidempotent is set 5019ec7b004SRick Macklem */ 5029ec7b004SRick Macklem if (nd->nd_repstat != NFSERR_DONTREPLY && 5039ec7b004SRick Macklem (rp->rc_refcnt > 0 || 5049ec7b004SRick Macklem ((nd->nd_flag & ND_SAVEREPLY) && (rp->rc_flag & RC_UDP)) || 5059ec7b004SRick Macklem ((nd->nd_flag & ND_SAVEREPLY) && !(rp->rc_flag & RC_UDP) && 5069ec7b004SRick Macklem nfsrc_tcpsavedreplies <= nfsrc_floodlevel && 5079ec7b004SRick Macklem nfsrc_tcpnonidempotent))) { 5089ec7b004SRick Macklem if (rp->rc_refcnt > 0) { 5099ec7b004SRick Macklem if (!(rp->rc_flag & RC_NFSV4)) 5109ec7b004SRick Macklem panic("update_cache refcnt"); 5119ec7b004SRick Macklem rp->rc_flag |= RC_REFCNT; 5129ec7b004SRick Macklem } 5139ec7b004SRick Macklem if ((nd->nd_flag & ND_NFSV2) && 5149ec7b004SRick Macklem nfsv2_repstat[newnfsv2_procid[nd->nd_procnum]]) { 5159ec7b004SRick Macklem rp->rc_status = nd->nd_repstat; 5169ec7b004SRick Macklem rp->rc_flag |= RC_REPSTATUS; 51793c5875bSRick Macklem mtx_unlock(mutex); 5189ec7b004SRick Macklem } else { 5199ec7b004SRick Macklem if (!(rp->rc_flag & RC_UDP)) { 52093c5875bSRick Macklem atomic_add_int(&nfsrc_tcpsavedreplies, 1); 5219ec7b004SRick Macklem if (nfsrc_tcpsavedreplies > 5221b819cf2SRick Macklem nfsstatsv1.srvcache_tcppeak) 5231b819cf2SRick Macklem nfsstatsv1.srvcache_tcppeak = 5249ec7b004SRick Macklem nfsrc_tcpsavedreplies; 5259ec7b004SRick Macklem } 52693c5875bSRick Macklem mtx_unlock(mutex); 527eb1b1807SGleb Smirnoff m = m_copym(nd->nd_mreq, 0, M_COPYALL, M_WAITOK); 52893c5875bSRick Macklem mtx_lock(mutex); 5294e22c98aSRick Macklem rp->rc_reply = m; 5309ec7b004SRick Macklem rp->rc_flag |= RC_REPMBUF; 53193c5875bSRick Macklem mtx_unlock(mutex); 5329ec7b004SRick Macklem } 5339ec7b004SRick Macklem if (rp->rc_flag & RC_UDP) { 5349ec7b004SRick Macklem rp->rc_timestamp = NFSD_MONOSEC + 5359ec7b004SRick Macklem NFSRVCACHE_UDPTIMEOUT; 5369ec7b004SRick Macklem nfsrc_unlock(rp); 5379ec7b004SRick Macklem } else { 53893c5875bSRick Macklem rp->rc_timestamp = NFSD_MONOSEC + nfsrc_tcptimeout; 5399ec7b004SRick Macklem if (rp->rc_refcnt > 0) 5409ec7b004SRick Macklem nfsrc_unlock(rp); 5419ec7b004SRick Macklem else 5429ec7b004SRick Macklem retrp = rp; 5439ec7b004SRick Macklem } 5449ec7b004SRick Macklem } else { 5459ec7b004SRick Macklem nfsrc_freecache(rp); 54693c5875bSRick Macklem mtx_unlock(mutex); 5479ec7b004SRick Macklem } 548a9285ae5SZack Kirsch 549a9285ae5SZack Kirsch out: 550a9285ae5SZack Kirsch NFSEXITCODE2(0, nd); 5519ec7b004SRick Macklem return (retrp); 5529ec7b004SRick Macklem } 5539ec7b004SRick Macklem 5549ec7b004SRick Macklem /* 5559ec7b004SRick Macklem * Invalidate and, if possible, free an in prog cache entry. 5569ec7b004SRick Macklem * Must not sleep. 5579ec7b004SRick Macklem */ 5589ec7b004SRick Macklem APPLESTATIC void 5599ec7b004SRick Macklem nfsrvd_delcache(struct nfsrvcache *rp) 5609ec7b004SRick Macklem { 56193c5875bSRick Macklem struct mtx *mutex; 5629ec7b004SRick Macklem 56393c5875bSRick Macklem mutex = nfsrc_cachemutex(rp); 5649ec7b004SRick Macklem if (!(rp->rc_flag & RC_INPROG)) 5659ec7b004SRick Macklem panic("nfsrvd_delcache not in prog"); 56693c5875bSRick Macklem mtx_lock(mutex); 5679ec7b004SRick Macklem rp->rc_flag &= ~RC_INPROG; 5689ec7b004SRick Macklem if (rp->rc_refcnt == 0 && !(rp->rc_flag & RC_LOCKED)) 5699ec7b004SRick Macklem nfsrc_freecache(rp); 57093c5875bSRick Macklem mtx_unlock(mutex); 5719ec7b004SRick Macklem } 5729ec7b004SRick Macklem 5739ec7b004SRick Macklem /* 5749ec7b004SRick Macklem * Called after nfsrvd_updatecache() once the reply is sent, to update 575d473bac7SAlexander Motin * the entry's sequence number and unlock it. The argument is 5769ec7b004SRick Macklem * the pointer returned by nfsrvd_updatecache(). 5779ec7b004SRick Macklem */ 5789ec7b004SRick Macklem APPLESTATIC void 5796103bae6SAlexander Motin nfsrvd_sentcache(struct nfsrvcache *rp, int have_seq, uint32_t seq) 5809ec7b004SRick Macklem { 581d473bac7SAlexander Motin struct nfsrchash_bucket *hbp; 5829ec7b004SRick Macklem 583d473bac7SAlexander Motin KASSERT(rp->rc_flag & RC_LOCKED, ("nfsrvd_sentcache not locked")); 5846103bae6SAlexander Motin if (have_seq) { 585d473bac7SAlexander Motin hbp = NFSRCAHASH(rp->rc_sockref); 586d473bac7SAlexander Motin mtx_lock(&hbp->mtx); 587d473bac7SAlexander Motin rp->rc_tcpseq = seq; 588d473bac7SAlexander Motin if (rp->rc_acked != RC_NO_ACK) 589d473bac7SAlexander Motin LIST_INSERT_HEAD(&hbp->tbl, rp, rc_ahash); 590d473bac7SAlexander Motin rp->rc_acked = RC_NO_ACK; 591d473bac7SAlexander Motin mtx_unlock(&hbp->mtx); 5926103bae6SAlexander Motin } 5939ec7b004SRick Macklem nfsrc_unlock(rp); 5949ec7b004SRick Macklem } 5959ec7b004SRick Macklem 5969ec7b004SRick Macklem /* 5979ec7b004SRick Macklem * Get a cache entry for TCP 5989ec7b004SRick Macklem * - key on <xid, nfs version> 5999ec7b004SRick Macklem * (allow multiple entries for a given key) 6009ec7b004SRick Macklem */ 6019ec7b004SRick Macklem static int 6029ec7b004SRick Macklem nfsrc_gettcp(struct nfsrv_descript *nd, struct nfsrvcache *newrp) 6039ec7b004SRick Macklem { 6049ec7b004SRick Macklem struct nfsrvcache *rp, *nextrp; 6059ec7b004SRick Macklem int i; 6069ec7b004SRick Macklem struct nfsrvcache *hitrp; 6079ec7b004SRick Macklem struct nfsrvhashhead *hp, nfsrc_templist; 6089ec7b004SRick Macklem int hit, ret = 0; 60993c5875bSRick Macklem struct mtx *mutex; 6109ec7b004SRick Macklem 61193c5875bSRick Macklem mutex = nfsrc_cachemutex(newrp); 6129ec7b004SRick Macklem hp = NFSRCHASH(newrp->rc_xid); 6139ec7b004SRick Macklem newrp->rc_reqlen = nfsrc_getlenandcksum(nd->nd_mrep, &newrp->rc_cksum); 6149ec7b004SRick Macklem tryagain: 61593c5875bSRick Macklem mtx_lock(mutex); 6169ec7b004SRick Macklem hit = 1; 6179ec7b004SRick Macklem LIST_INIT(&nfsrc_templist); 6189ec7b004SRick Macklem /* 6199ec7b004SRick Macklem * Get all the matches and put them on the temp list. 6209ec7b004SRick Macklem */ 6219ec7b004SRick Macklem rp = LIST_FIRST(hp); 6229ec7b004SRick Macklem while (rp != LIST_END(hp)) { 6239ec7b004SRick Macklem nextrp = LIST_NEXT(rp, rc_hash); 6249ec7b004SRick Macklem if (newrp->rc_xid == rp->rc_xid && 6259ec7b004SRick Macklem (!(rp->rc_flag & RC_INPROG) || 6269ec7b004SRick Macklem ((newrp->rc_flag & RC_SAMETCPCONN) && 6279ec7b004SRick Macklem newrp->rc_sockref == rp->rc_sockref)) && 6289ec7b004SRick Macklem (newrp->rc_flag & rp->rc_flag & RC_NFSVERS) && 6299ec7b004SRick Macklem newrp->rc_proc == rp->rc_proc && 6309ec7b004SRick Macklem ((newrp->rc_flag & RC_NFSV4) && 6319ec7b004SRick Macklem newrp->rc_sockref != rp->rc_sockref && 6329ec7b004SRick Macklem newrp->rc_cachetime >= rp->rc_cachetime) 6339ec7b004SRick Macklem && newrp->rc_reqlen == rp->rc_reqlen && 6349ec7b004SRick Macklem newrp->rc_cksum == rp->rc_cksum) { 6359ec7b004SRick Macklem LIST_REMOVE(rp, rc_hash); 6369ec7b004SRick Macklem LIST_INSERT_HEAD(&nfsrc_templist, rp, rc_hash); 6379ec7b004SRick Macklem } 6389ec7b004SRick Macklem rp = nextrp; 6399ec7b004SRick Macklem } 6409ec7b004SRick Macklem 6419ec7b004SRick Macklem /* 6429ec7b004SRick Macklem * Now, use nfsrc_templist to decide if there is a match. 6439ec7b004SRick Macklem */ 6449ec7b004SRick Macklem i = 0; 6459ec7b004SRick Macklem LIST_FOREACH(rp, &nfsrc_templist, rc_hash) { 6469ec7b004SRick Macklem i++; 6479ec7b004SRick Macklem if (rp->rc_refcnt > 0) { 6489ec7b004SRick Macklem hit = 0; 6499ec7b004SRick Macklem break; 6509ec7b004SRick Macklem } 6519ec7b004SRick Macklem } 6529ec7b004SRick Macklem /* 6539ec7b004SRick Macklem * Can be a hit only if one entry left. 6549ec7b004SRick Macklem * Note possible hit entry and put nfsrc_templist back on hash 6559ec7b004SRick Macklem * list. 6569ec7b004SRick Macklem */ 6579ec7b004SRick Macklem if (i != 1) 6589ec7b004SRick Macklem hit = 0; 6599ec7b004SRick Macklem hitrp = rp = LIST_FIRST(&nfsrc_templist); 6609ec7b004SRick Macklem while (rp != LIST_END(&nfsrc_templist)) { 6619ec7b004SRick Macklem nextrp = LIST_NEXT(rp, rc_hash); 6629ec7b004SRick Macklem LIST_REMOVE(rp, rc_hash); 6639ec7b004SRick Macklem LIST_INSERT_HEAD(hp, rp, rc_hash); 6649ec7b004SRick Macklem rp = nextrp; 6659ec7b004SRick Macklem } 6669ec7b004SRick Macklem if (LIST_FIRST(&nfsrc_templist) != LIST_END(&nfsrc_templist)) 6679ec7b004SRick Macklem panic("nfs gettcp cache templist"); 6689ec7b004SRick Macklem 6699ec7b004SRick Macklem if (hit) { 6709ec7b004SRick Macklem rp = hitrp; 6719ec7b004SRick Macklem if ((rp->rc_flag & RC_LOCKED) != 0) { 6729ec7b004SRick Macklem rp->rc_flag |= RC_WANTED; 67393c5875bSRick Macklem (void)mtx_sleep(rp, mutex, (PZERO - 1) | PDROP, 67493c5875bSRick Macklem "nfsrc", 10 * hz); 6759ec7b004SRick Macklem goto tryagain; 6769ec7b004SRick Macklem } 6779ec7b004SRick Macklem if (rp->rc_flag == 0) 6789ec7b004SRick Macklem panic("nfs tcp cache0"); 6799ec7b004SRick Macklem rp->rc_flag |= RC_LOCKED; 6809ec7b004SRick Macklem if (rp->rc_flag & RC_INPROG) { 6811b819cf2SRick Macklem nfsstatsv1.srvcache_inproghits++; 68293c5875bSRick Macklem mtx_unlock(mutex); 6839ec7b004SRick Macklem if (newrp->rc_sockref == rp->rc_sockref) 6849ec7b004SRick Macklem nfsrc_marksametcpconn(rp->rc_sockref); 6859ec7b004SRick Macklem ret = RC_DROPIT; 6869ec7b004SRick Macklem } else if (rp->rc_flag & RC_REPSTATUS) { 6879ec7b004SRick Macklem /* 6889ec7b004SRick Macklem * V2 only. 6899ec7b004SRick Macklem */ 6901b819cf2SRick Macklem nfsstatsv1.srvcache_nonidemdonehits++; 69193c5875bSRick Macklem mtx_unlock(mutex); 6929ec7b004SRick Macklem if (newrp->rc_sockref == rp->rc_sockref) 6939ec7b004SRick Macklem nfsrc_marksametcpconn(rp->rc_sockref); 6949ec7b004SRick Macklem ret = RC_REPLY; 6959ec7b004SRick Macklem nfsrvd_rephead(nd); 6969ec7b004SRick Macklem *(nd->nd_errp) = rp->rc_status; 69793c5875bSRick Macklem rp->rc_timestamp = NFSD_MONOSEC + nfsrc_tcptimeout; 6989ec7b004SRick Macklem } else if (rp->rc_flag & RC_REPMBUF) { 6991b819cf2SRick Macklem nfsstatsv1.srvcache_nonidemdonehits++; 70093c5875bSRick Macklem mtx_unlock(mutex); 7019ec7b004SRick Macklem if (newrp->rc_sockref == rp->rc_sockref) 7029ec7b004SRick Macklem nfsrc_marksametcpconn(rp->rc_sockref); 7039ec7b004SRick Macklem ret = RC_REPLY; 7049ec7b004SRick Macklem nd->nd_mreq = m_copym(rp->rc_reply, 0, 705eb1b1807SGleb Smirnoff M_COPYALL, M_WAITOK); 70693c5875bSRick Macklem rp->rc_timestamp = NFSD_MONOSEC + nfsrc_tcptimeout; 7079ec7b004SRick Macklem } else { 7089ec7b004SRick Macklem panic("nfs tcp cache1"); 7099ec7b004SRick Macklem } 7109ec7b004SRick Macklem nfsrc_unlock(rp); 7119ec7b004SRick Macklem free((caddr_t)newrp, M_NFSRVCACHE); 712a9285ae5SZack Kirsch goto out; 7139ec7b004SRick Macklem } 7141b819cf2SRick Macklem nfsstatsv1.srvcache_misses++; 7151b819cf2SRick Macklem atomic_add_int(&nfsstatsv1.srvcache_size, 1); 7169ec7b004SRick Macklem 7179ec7b004SRick Macklem /* 7189ec7b004SRick Macklem * For TCP, multiple entries for a key are allowed, so don't 7199ec7b004SRick Macklem * chain it into the hash table until done. 7209ec7b004SRick Macklem */ 7219ec7b004SRick Macklem newrp->rc_cachetime = NFSD_MONOSEC; 7229ec7b004SRick Macklem newrp->rc_flag |= RC_INPROG; 7239ec7b004SRick Macklem LIST_INSERT_HEAD(hp, newrp, rc_hash); 72493c5875bSRick Macklem mtx_unlock(mutex); 7259ec7b004SRick Macklem nd->nd_rp = newrp; 726a9285ae5SZack Kirsch ret = RC_DOIT; 727a9285ae5SZack Kirsch 728a9285ae5SZack Kirsch out: 729a9285ae5SZack Kirsch NFSEXITCODE2(0, nd); 730a9285ae5SZack Kirsch return (ret); 7319ec7b004SRick Macklem } 7329ec7b004SRick Macklem 7339ec7b004SRick Macklem /* 7349ec7b004SRick Macklem * Lock a cache entry. 7359ec7b004SRick Macklem */ 7369ec7b004SRick Macklem static void 7379ec7b004SRick Macklem nfsrc_lock(struct nfsrvcache *rp) 7389ec7b004SRick Macklem { 73993c5875bSRick Macklem struct mtx *mutex; 74093c5875bSRick Macklem 74193c5875bSRick Macklem mutex = nfsrc_cachemutex(rp); 74293c5875bSRick Macklem mtx_assert(mutex, MA_OWNED); 7439ec7b004SRick Macklem while ((rp->rc_flag & RC_LOCKED) != 0) { 7449ec7b004SRick Macklem rp->rc_flag |= RC_WANTED; 74593c5875bSRick Macklem (void)mtx_sleep(rp, mutex, PZERO - 1, "nfsrc", 0); 7469ec7b004SRick Macklem } 7479ec7b004SRick Macklem rp->rc_flag |= RC_LOCKED; 7489ec7b004SRick Macklem } 7499ec7b004SRick Macklem 7509ec7b004SRick Macklem /* 7519ec7b004SRick Macklem * Unlock a cache entry. 7529ec7b004SRick Macklem */ 7539ec7b004SRick Macklem static void 7549ec7b004SRick Macklem nfsrc_unlock(struct nfsrvcache *rp) 7559ec7b004SRick Macklem { 75693c5875bSRick Macklem struct mtx *mutex; 7574e22c98aSRick Macklem 75893c5875bSRick Macklem mutex = nfsrc_cachemutex(rp); 75993c5875bSRick Macklem mtx_lock(mutex); 7609ec7b004SRick Macklem rp->rc_flag &= ~RC_LOCKED; 7619ec7b004SRick Macklem nfsrc_wanted(rp); 76293c5875bSRick Macklem mtx_unlock(mutex); 7639ec7b004SRick Macklem } 7649ec7b004SRick Macklem 7659ec7b004SRick Macklem /* 7669ec7b004SRick Macklem * Wakeup anyone wanting entry. 7679ec7b004SRick Macklem */ 7689ec7b004SRick Macklem static void 7699ec7b004SRick Macklem nfsrc_wanted(struct nfsrvcache *rp) 7709ec7b004SRick Macklem { 7719ec7b004SRick Macklem if (rp->rc_flag & RC_WANTED) { 7729ec7b004SRick Macklem rp->rc_flag &= ~RC_WANTED; 7739ec7b004SRick Macklem wakeup((caddr_t)rp); 7749ec7b004SRick Macklem } 7759ec7b004SRick Macklem } 7769ec7b004SRick Macklem 7779ec7b004SRick Macklem /* 7789ec7b004SRick Macklem * Free up the entry. 7799ec7b004SRick Macklem * Must not sleep. 7809ec7b004SRick Macklem */ 7819ec7b004SRick Macklem static void 7829ec7b004SRick Macklem nfsrc_freecache(struct nfsrvcache *rp) 7839ec7b004SRick Macklem { 784d473bac7SAlexander Motin struct nfsrchash_bucket *hbp; 7859ec7b004SRick Macklem 7869ec7b004SRick Macklem LIST_REMOVE(rp, rc_hash); 7879ec7b004SRick Macklem if (rp->rc_flag & RC_UDP) { 7889ec7b004SRick Macklem TAILQ_REMOVE(&nfsrvudplru, rp, rc_lru); 7899ec7b004SRick Macklem nfsrc_udpcachesize--; 790d473bac7SAlexander Motin } else if (rp->rc_acked != RC_NO_SEQ) { 791d473bac7SAlexander Motin hbp = NFSRCAHASH(rp->rc_sockref); 792d473bac7SAlexander Motin mtx_lock(&hbp->mtx); 793d473bac7SAlexander Motin if (rp->rc_acked == RC_NO_ACK) 794d473bac7SAlexander Motin LIST_REMOVE(rp, rc_ahash); 795d473bac7SAlexander Motin mtx_unlock(&hbp->mtx); 7969ec7b004SRick Macklem } 7979ec7b004SRick Macklem nfsrc_wanted(rp); 7989ec7b004SRick Macklem if (rp->rc_flag & RC_REPMBUF) { 7999ec7b004SRick Macklem mbuf_freem(rp->rc_reply); 8009ec7b004SRick Macklem if (!(rp->rc_flag & RC_UDP)) 80193c5875bSRick Macklem atomic_add_int(&nfsrc_tcpsavedreplies, -1); 8029ec7b004SRick Macklem } 8039ec7b004SRick Macklem FREE((caddr_t)rp, M_NFSRVCACHE); 8041b819cf2SRick Macklem atomic_add_int(&nfsstatsv1.srvcache_size, -1); 8059ec7b004SRick Macklem } 8069ec7b004SRick Macklem 8079ec7b004SRick Macklem /* 80852776c50SZack Kirsch * Clean out the cache. Called when nfsserver module is unloaded. 8099ec7b004SRick Macklem */ 8109ec7b004SRick Macklem APPLESTATIC void 8119ec7b004SRick Macklem nfsrvd_cleancache(void) 8129ec7b004SRick Macklem { 8139ec7b004SRick Macklem struct nfsrvcache *rp, *nextrp; 8149ec7b004SRick Macklem int i; 8159ec7b004SRick Macklem 8169ec7b004SRick Macklem for (i = 0; i < NFSRVCACHE_HASHSIZE; i++) { 81793c5875bSRick Macklem mtx_lock(&nfsrchash_table[i].mtx); 81893c5875bSRick Macklem LIST_FOREACH_SAFE(rp, &nfsrchash_table[i].tbl, rc_hash, nextrp) 8199ec7b004SRick Macklem nfsrc_freecache(rp); 82093c5875bSRick Macklem mtx_unlock(&nfsrchash_table[i].mtx); 8219ec7b004SRick Macklem } 82293c5875bSRick Macklem mtx_lock(&nfsrc_udpmtx); 8239ec7b004SRick Macklem for (i = 0; i < NFSRVCACHE_HASHSIZE; i++) { 8249ec7b004SRick Macklem LIST_FOREACH_SAFE(rp, &nfsrvudphashtbl[i], rc_hash, nextrp) { 8259ec7b004SRick Macklem nfsrc_freecache(rp); 8269ec7b004SRick Macklem } 8279ec7b004SRick Macklem } 8281b819cf2SRick Macklem nfsstatsv1.srvcache_size = 0; 82993c5875bSRick Macklem mtx_unlock(&nfsrc_udpmtx); 8309ec7b004SRick Macklem nfsrc_tcpsavedreplies = 0; 8319ec7b004SRick Macklem } 8329ec7b004SRick Macklem 8331555cf04SAlexander Motin #define HISTSIZE 16 8349ec7b004SRick Macklem /* 8359ec7b004SRick Macklem * The basic rule is to get rid of entries that are expired. 8369ec7b004SRick Macklem */ 837d473bac7SAlexander Motin void 838d473bac7SAlexander Motin nfsrc_trimcache(u_int64_t sockref, uint32_t snd_una, int final) 8399ec7b004SRick Macklem { 840d473bac7SAlexander Motin struct nfsrchash_bucket *hbp; 8419ec7b004SRick Macklem struct nfsrvcache *rp, *nextrp; 842d473bac7SAlexander Motin int force, lastslot, i, j, k, tto, time_histo[HISTSIZE]; 84393c5875bSRick Macklem time_t thisstamp; 84493c5875bSRick Macklem static time_t udp_lasttrim = 0, tcp_lasttrim = 0; 845d473bac7SAlexander Motin static int onethread = 0, oneslot = 0; 846d473bac7SAlexander Motin 847d473bac7SAlexander Motin if (sockref != 0) { 848d473bac7SAlexander Motin hbp = NFSRCAHASH(sockref); 849d473bac7SAlexander Motin mtx_lock(&hbp->mtx); 850d473bac7SAlexander Motin LIST_FOREACH_SAFE(rp, &hbp->tbl, rc_ahash, nextrp) { 851d473bac7SAlexander Motin if (sockref == rp->rc_sockref) { 852d473bac7SAlexander Motin if (SEQ_GEQ(snd_una, rp->rc_tcpseq)) { 853d473bac7SAlexander Motin rp->rc_acked = RC_ACK; 854d473bac7SAlexander Motin LIST_REMOVE(rp, rc_ahash); 855d473bac7SAlexander Motin } else if (final) { 856d473bac7SAlexander Motin rp->rc_acked = RC_NACK; 857d473bac7SAlexander Motin LIST_REMOVE(rp, rc_ahash); 858d473bac7SAlexander Motin } 859d473bac7SAlexander Motin } 860d473bac7SAlexander Motin } 861d473bac7SAlexander Motin mtx_unlock(&hbp->mtx); 862d473bac7SAlexander Motin } 8639ec7b004SRick Macklem 86493c5875bSRick Macklem if (atomic_cmpset_acq_int(&onethread, 0, 1) == 0) 86593c5875bSRick Macklem return; 86693c5875bSRick Macklem if (NFSD_MONOSEC != udp_lasttrim || 86793c5875bSRick Macklem nfsrc_udpcachesize >= (nfsrc_udphighwater + 86893c5875bSRick Macklem nfsrc_udphighwater / 2)) { 86993c5875bSRick Macklem mtx_lock(&nfsrc_udpmtx); 87093c5875bSRick Macklem udp_lasttrim = NFSD_MONOSEC; 8719ec7b004SRick Macklem TAILQ_FOREACH_SAFE(rp, &nfsrvudplru, rc_lru, nextrp) { 8729ec7b004SRick Macklem if (!(rp->rc_flag & (RC_INPROG|RC_LOCKED|RC_WANTED)) 8739ec7b004SRick Macklem && rp->rc_refcnt == 0 8749ec7b004SRick Macklem && ((rp->rc_flag & RC_REFCNT) || 87593c5875bSRick Macklem udp_lasttrim > rp->rc_timestamp || 8769ec7b004SRick Macklem nfsrc_udpcachesize > nfsrc_udphighwater)) 8779ec7b004SRick Macklem nfsrc_freecache(rp); 8789ec7b004SRick Macklem } 87993c5875bSRick Macklem mtx_unlock(&nfsrc_udpmtx); 88093c5875bSRick Macklem } 88193c5875bSRick Macklem if (NFSD_MONOSEC != tcp_lasttrim || 88293c5875bSRick Macklem nfsrc_tcpsavedreplies >= nfsrc_tcphighwater) { 883d473bac7SAlexander Motin force = nfsrc_tcphighwater / 4; 884d473bac7SAlexander Motin if (force > 0 && 885d473bac7SAlexander Motin nfsrc_tcpsavedreplies + force >= nfsrc_tcphighwater) { 8861555cf04SAlexander Motin for (i = 0; i < HISTSIZE; i++) 88793c5875bSRick Macklem time_histo[i] = 0; 888d473bac7SAlexander Motin i = 0; 88945e18ea7SAlexander Motin lastslot = NFSRVCACHE_HASHSIZE - 1; 890d473bac7SAlexander Motin } else { 891d473bac7SAlexander Motin force = 0; 892d473bac7SAlexander Motin if (NFSD_MONOSEC != tcp_lasttrim) { 893d473bac7SAlexander Motin i = 0; 894d473bac7SAlexander Motin lastslot = NFSRVCACHE_HASHSIZE - 1; 895d473bac7SAlexander Motin } else { 896d473bac7SAlexander Motin lastslot = i = oneslot; 897d473bac7SAlexander Motin if (++oneslot >= NFSRVCACHE_HASHSIZE) 898d473bac7SAlexander Motin oneslot = 0; 899d473bac7SAlexander Motin } 900d473bac7SAlexander Motin } 9011555cf04SAlexander Motin tto = nfsrc_tcptimeout; 90293c5875bSRick Macklem tcp_lasttrim = NFSD_MONOSEC; 903d473bac7SAlexander Motin for (; i <= lastslot; i++) { 904d473bac7SAlexander Motin mtx_lock(&nfsrchash_table[i].mtx); 90593c5875bSRick Macklem LIST_FOREACH_SAFE(rp, &nfsrchash_table[i].tbl, rc_hash, 90693c5875bSRick Macklem nextrp) { 90793c5875bSRick Macklem if (!(rp->rc_flag & 90893c5875bSRick Macklem (RC_INPROG|RC_LOCKED|RC_WANTED)) 90993c5875bSRick Macklem && rp->rc_refcnt == 0) { 9101555cf04SAlexander Motin if ((rp->rc_flag & RC_REFCNT) || 9111555cf04SAlexander Motin tcp_lasttrim > rp->rc_timestamp || 912d473bac7SAlexander Motin rp->rc_acked == RC_ACK) { 9131555cf04SAlexander Motin nfsrc_freecache(rp); 9141555cf04SAlexander Motin continue; 9151555cf04SAlexander Motin } 9161555cf04SAlexander Motin 917d473bac7SAlexander Motin if (force == 0) 9181555cf04SAlexander Motin continue; 91993c5875bSRick Macklem /* 92093c5875bSRick Macklem * The timestamps range from roughly the 92193c5875bSRick Macklem * present (tcp_lasttrim) to the present 92293c5875bSRick Macklem * + nfsrc_tcptimeout. Generate a simple 92393c5875bSRick Macklem * histogram of where the timeouts fall. 92493c5875bSRick Macklem */ 92593c5875bSRick Macklem j = rp->rc_timestamp - tcp_lasttrim; 9261555cf04SAlexander Motin if (j >= tto) 9271555cf04SAlexander Motin j = HISTSIZE - 1; 9281555cf04SAlexander Motin else if (j < 0) 92993c5875bSRick Macklem j = 0; 9301555cf04SAlexander Motin else 9311555cf04SAlexander Motin j = j * HISTSIZE / tto; 93293c5875bSRick Macklem time_histo[j]++; 9339ec7b004SRick Macklem } 9349ec7b004SRick Macklem } 93593c5875bSRick Macklem mtx_unlock(&nfsrchash_table[i].mtx); 93693c5875bSRick Macklem } 937d473bac7SAlexander Motin if (force) { 93893c5875bSRick Macklem /* 93993c5875bSRick Macklem * Trim some more with a smaller timeout of as little 94093c5875bSRick Macklem * as 20% of nfsrc_tcptimeout to try and get below 94193c5875bSRick Macklem * 80% of the nfsrc_tcphighwater. 94293c5875bSRick Macklem */ 94393c5875bSRick Macklem k = 0; 9441555cf04SAlexander Motin for (i = 0; i < (HISTSIZE - 2); i++) { 94593c5875bSRick Macklem k += time_histo[i]; 946d473bac7SAlexander Motin if (k > force) 94793c5875bSRick Macklem break; 94893c5875bSRick Macklem } 9491555cf04SAlexander Motin k = tto * (i + 1) / HISTSIZE; 95093c5875bSRick Macklem if (k < 1) 95193c5875bSRick Macklem k = 1; 95293c5875bSRick Macklem thisstamp = tcp_lasttrim + k; 95393c5875bSRick Macklem for (i = 0; i < NFSRVCACHE_HASHSIZE; i++) { 95493c5875bSRick Macklem mtx_lock(&nfsrchash_table[i].mtx); 95593c5875bSRick Macklem LIST_FOREACH_SAFE(rp, &nfsrchash_table[i].tbl, 95693c5875bSRick Macklem rc_hash, nextrp) { 95793c5875bSRick Macklem if (!(rp->rc_flag & 95893c5875bSRick Macklem (RC_INPROG|RC_LOCKED|RC_WANTED)) 95993c5875bSRick Macklem && rp->rc_refcnt == 0 96093c5875bSRick Macklem && ((rp->rc_flag & RC_REFCNT) || 96193c5875bSRick Macklem thisstamp > rp->rc_timestamp || 962d473bac7SAlexander Motin rp->rc_acked == RC_ACK)) 96393c5875bSRick Macklem nfsrc_freecache(rp); 96493c5875bSRick Macklem } 96593c5875bSRick Macklem mtx_unlock(&nfsrchash_table[i].mtx); 96693c5875bSRick Macklem } 96793c5875bSRick Macklem } 96893c5875bSRick Macklem } 96993c5875bSRick Macklem atomic_store_rel_int(&onethread, 0); 9709ec7b004SRick Macklem } 9719ec7b004SRick Macklem 9729ec7b004SRick Macklem /* 9739ec7b004SRick Macklem * Add a seqid# reference to the cache entry. 9749ec7b004SRick Macklem */ 9759ec7b004SRick Macklem APPLESTATIC void 9769ec7b004SRick Macklem nfsrvd_refcache(struct nfsrvcache *rp) 9779ec7b004SRick Macklem { 97893c5875bSRick Macklem struct mtx *mutex; 9799ec7b004SRick Macklem 980c59e4cc3SRick Macklem if (rp == NULL) 981c59e4cc3SRick Macklem /* For NFSv4.1, there is no cache entry. */ 982c59e4cc3SRick Macklem return; 98393c5875bSRick Macklem mutex = nfsrc_cachemutex(rp); 98493c5875bSRick Macklem mtx_lock(mutex); 9859ec7b004SRick Macklem if (rp->rc_refcnt < 0) 9869ec7b004SRick Macklem panic("nfs cache refcnt"); 9879ec7b004SRick Macklem rp->rc_refcnt++; 98893c5875bSRick Macklem mtx_unlock(mutex); 9899ec7b004SRick Macklem } 9909ec7b004SRick Macklem 9919ec7b004SRick Macklem /* 9929ec7b004SRick Macklem * Dereference a seqid# cache entry. 9939ec7b004SRick Macklem */ 9949ec7b004SRick Macklem APPLESTATIC void 9959ec7b004SRick Macklem nfsrvd_derefcache(struct nfsrvcache *rp) 9969ec7b004SRick Macklem { 99793c5875bSRick Macklem struct mtx *mutex; 9989ec7b004SRick Macklem 99993c5875bSRick Macklem mutex = nfsrc_cachemutex(rp); 100093c5875bSRick Macklem mtx_lock(mutex); 10019ec7b004SRick Macklem if (rp->rc_refcnt <= 0) 10029ec7b004SRick Macklem panic("nfs cache derefcnt"); 10039ec7b004SRick Macklem rp->rc_refcnt--; 10049ec7b004SRick Macklem if (rp->rc_refcnt == 0 && !(rp->rc_flag & (RC_LOCKED | RC_INPROG))) 10059ec7b004SRick Macklem nfsrc_freecache(rp); 100693c5875bSRick Macklem mtx_unlock(mutex); 10079ec7b004SRick Macklem } 10089ec7b004SRick Macklem 10099ec7b004SRick Macklem /* 10109ec7b004SRick Macklem * Calculate the length of the mbuf list and a checksum on the first up to 10119ec7b004SRick Macklem * NFSRVCACHE_CHECKLEN bytes. 10129ec7b004SRick Macklem */ 10139ec7b004SRick Macklem static int 10149ec7b004SRick Macklem nfsrc_getlenandcksum(mbuf_t m1, u_int16_t *cksum) 10159ec7b004SRick Macklem { 10169ec7b004SRick Macklem int len = 0, cklen; 10179ec7b004SRick Macklem mbuf_t m; 10189ec7b004SRick Macklem 10199ec7b004SRick Macklem m = m1; 10209ec7b004SRick Macklem while (m) { 10219ec7b004SRick Macklem len += mbuf_len(m); 10229ec7b004SRick Macklem m = mbuf_next(m); 10239ec7b004SRick Macklem } 10249ec7b004SRick Macklem cklen = (len > NFSRVCACHE_CHECKLEN) ? NFSRVCACHE_CHECKLEN : len; 10259ec7b004SRick Macklem *cksum = in_cksum(m1, cklen); 10269ec7b004SRick Macklem return (len); 10279ec7b004SRick Macklem } 10289ec7b004SRick Macklem 10299ec7b004SRick Macklem /* 10309ec7b004SRick Macklem * Mark a TCP connection that is seeing retries. Should never happen for 10319ec7b004SRick Macklem * NFSv4. 10329ec7b004SRick Macklem */ 10339ec7b004SRick Macklem static void 10349ec7b004SRick Macklem nfsrc_marksametcpconn(u_int64_t sockref) 10359ec7b004SRick Macklem { 10369ec7b004SRick Macklem } 10379ec7b004SRick Macklem 1038