chiefnerd 18 Posted January 16, 2017 Share Posted January 16, 2017 Really don't know if this is the right place to post things. I'm streamlining emby for speed as much as possible so I've been running some image optimisation via imagemagick and optipng but I just want to know 100% where emby is serving images from to Emby for Kodi. Are they being served direct from the media folder itself or from one of the folders in the emby cache? I really don't want to optimise all images only to find I've been doing it in completely the wrong place. Same in the Web Interface- is Emby serving and resizing from the media folders or is it caching resized images somewhere like it's cache folder? I'm not awkward BTW, I just love speed lol Link to comment Share on other sites More sharing options...
Luke 37994 Posted January 16, 2017 Share Posted January 16, 2017 From wherever you configured your metadata to save to. If any resizing is requested by the apps then those are cached internally for future re-use. Link to comment Share on other sites More sharing options...
chiefnerd 18 Posted January 16, 2017 Author Share Posted January 16, 2017 That's great...is there anyway of forcing that cache to rebuild itself from the new resized files in the media folders? Link to comment Share on other sites More sharing options...
Luke 37994 Posted January 16, 2017 Share Posted January 16, 2017 It will. There is nothing you need to do. Link to comment Share on other sites More sharing options...
chiefnerd 18 Posted January 16, 2017 Author Share Posted January 16, 2017 Great stuff...resizing things should help endlessly with the Kodi addon on low powered devices where memory and read/write are are an issue. Amazing how fast you can get things going with Nginx sitting in front and caching properly to the backend....managed an A across the board on webpage tests which is damned hard on any website so well done. That's with Nginx sitting as reverse proxy and Cloudflare as a CDN. Link to comment Share on other sites More sharing options...
pir8radio 1301 Posted January 17, 2017 Share Posted January 17, 2017 Amazing how fast you can get things going with Nginx sitting in front and caching properly to the backend....managed an A across the board on webpage tests which is damned hard on any website so well done. That's with Nginx sitting as reverse proxy and Cloudflare as a CDN. I did not think the CDN's could cache a page they don't have access too? On my emby server around 95% of everything viewed in my browser is/should be cached LOCALLY on the browser machine after your initial view.. Javascript, CSS, images etc, majority of which is shared across all emby pages... The below screen capture is what my client browser see's when loading, most of the content is loaded from my client side html5 storage. Browser caching will always be faster than any server side cache because you are not transferring those files over the network connection. But anyway, be careful with CDN's and nginx caching, if not setup correctly you can cache an admin session and allow a regular user to hijack your account by accident. I guess mine didn't do "horrible" using https://tools.pingdom.com/ to test. BTW how do you like Cloudflare? Link to comment Share on other sites More sharing options...
chiefnerd 18 Posted January 17, 2017 Author Share Posted January 17, 2017 (edited) They aren't caching pages, they're serving requests locally. I'm a fan of cloudflare but some aren't, easy to manage external firewall and webapp firewall, protects your actual IP if setup correctly, free SSL\z caches pretty well out of the box and if you pay for things there,s an awful lot more. Screen attached serving from 2 x Intel Xeon E5 2640v4 -192 GB -18 TB RAID - 1Gbit/sec Edited January 17, 2017 by chiefnerd Link to comment Share on other sites More sharing options...
pir8radio 1301 Posted January 17, 2017 Share Posted January 17, 2017 (edited) Ill have to play around with cloudflare.. Edited January 17, 2017 by pir8radio Link to comment Share on other sites More sharing options...
chiefnerd 18 Posted January 17, 2017 Author Share Posted January 17, 2017 https://www.webpagetest.org/ Used it for years for all websites- gives a good indication of issues, tests on multiple devices from multiple locations and also has great great user forums for the different servers and how best to overcome issues. Just choose a location nearest your main core of site visitors. Link to comment Share on other sites More sharing options...
pir8radio 1301 Posted January 17, 2017 Share Posted January 17, 2017 Cool, Ill try cloudflare! I've been reading, and it does look like the CDN caches static content on the edge servers like I thought.. So ill setup and give it a try. "We are able to cache the static files on your site like images and CSS with our free CDN. Our system is smart enough to differentiate between static and dynamic content" Link to comment Share on other sites More sharing options...
chiefnerd 18 Posted January 17, 2017 Author Share Posted January 17, 2017 (edited) To get SSL caching over cloudflare use their own origin certs - it secures you from their edge to your server, it can then be de-encrypted in edge, files cached ready for the next request. Minimal SSL overhead as it only needs to de-encrpyt x requests to check cache validity. Also, although smart you always need to set your own cache headers, those will then be passed onto browsers etc as well. If you do that just don't choose strict SLL. you need your own certs for that setting Edited January 18, 2017 by chiefnerd Link to comment Share on other sites More sharing options...
pir8radio 1301 Posted January 19, 2017 Share Posted January 19, 2017 (edited) After some education and testing. a CDN like cloudflare is nice, but can actually slow things down for some. Cloudflare CDN actually proxies all traffic including non-cached traffic (actual media streams different sized images etc). If you use DNS only then you loose all of the benefits of using a CDN. So what happens is CDN cached images and what not load slower on the first view, then get cached on the CDN and are fast from there on out. Unless you resize your browser window then you have to slow load and re-cache those sized images. Where the CDN kills things is the stream, cloudflare wont cache the media stream, so now your end users have to rely on cloudflares backbone link to transfer the streams to the edge servers, then to your users. Even if someone is closer to your origin server they will be using the closest cloudflare edge server which may be further away. So the "website tests" may look good for static code/images, streaming and dynamic content will render slower than without a CDN. I'm going to continue to use it for a while to see just how much the streams slow, and if cloudflare kicks me off the free CDN from so much traffic usage lol.. Two days of using them as my CDN and I have pushed 23 gigs through their system. Edited January 19, 2017 by pir8radio Link to comment Share on other sites More sharing options...
chiefnerd 18 Posted January 19, 2017 Author Share Posted January 19, 2017 (edited) Wrong on almost all accounts but hey ho, what would I know, we only host about 1300 websites on our servers, all nginx and all via cloudflare for cdn delivery because it lessens server load, server bandwidth and provides excellent security. They provide a hell of alot more than simply delivering some content and if you've ever had to mitigate ddos attacks you'd know what I'm talking about. I also hate to burst your bubble but 23gb in a day is absolutely nothing - attached stats from a quiet server over 24 hours Edited January 19, 2017 by chiefnerd Link to comment Share on other sites More sharing options...
pir8radio 1301 Posted January 19, 2017 Share Posted January 19, 2017 Wrong on almost all accounts but hey ho, what would I know, we only host about 1300 websites on our servers, all nginx and all via cloudflare for cdn delivery because it lessens server load, server bandwidth and provides excellent security. They provide a hell of alot more than simply delivering some content and if you've ever had to mitigate ddos attacks you'd know what I'm talking about. I also hate to burst your bubble but 23gb in a day is absolutely nothing. Yes it lessens server load for static content. But for emby you will find that most content is not cached and has the potential to slow media streams. Images it should work great. For example below is a user watching a movie on emby. It's not cached, and the only way they could know how much bandwidth that movie consumed is if they were acting as a proxy (which they absolutely are) so that movie has to pass from origin to edge to user, and I have no control of cloudflares backbone bandwidth or Qos. Maybe you use the enterprise version and it does things differently, I'm talking entirely the free version here. All logs show cloudflare is acting as a proxy there is nothing directly pulled from my origin servers. Plus they say that they are a proxy here "CloudFlare acts as a reverse proxy, a user's browser connects to the CloudFlare servers which then request the content from the host server on behalf of the user." Link to comment Share on other sites More sharing options...
chiefnerd 18 Posted January 19, 2017 Author Share Posted January 19, 2017 Your logs are wrong and you probably don't have cloudflare enabled in nginx giving wrong values. CloudFlare free and enterprise work exactly the same way, business and enterprise just have far more options to customise and none cache video. Vixei delivery via CDN is very expensive and tbe realm if akamai and the large commercial content CDNs. Link to comment Share on other sites More sharing options...
pir8radio 1301 Posted January 19, 2017 Share Posted January 19, 2017 Your logs are wrong and you probably don't have cloudflare enabled in nginx giving wrong values. CloudFlare free and enterprise work exactly the same way, business and enterprise just have far more options to customise and none cache video. Vixei delivery via CDN is very expensive and tbe realm if akamai and the large commercial content CDNs. Possible... But then I guess cloudflare support is wrong too: https://support.cloudflare.com/hc/en-us/articles/200169706-Can-I-use-CloudFlare-with-a-streaming-music-or-video-site- Link to comment Share on other sites More sharing options...
chiefnerd 18 Posted January 19, 2017 Author Share Posted January 19, 2017 If you have cloudflare caching video you've made a huge cockup somewhere unless you simply don't understand how they're presenting their logs. Check your own server logs and you'll see all the requests. And I'll repeat this - we have around 1300 sites going through cloudflare, I don't need you to post bloody FAQs son, I am more than aware of a exactly how cloudflare works and you're simply teaching a Granny how to suck eggs. Do you understand that the server Emby is running on is just 1 of our servers, we have 20 dedicated servers some slightly less spec, some double the spec for transcoding live IPTV. Abd all of our content goes via cloudflare to protect origin servers. Link to comment Share on other sites More sharing options...
pir8radio 1301 Posted January 19, 2017 Share Posted January 19, 2017 No worries... No need to get worked up.. I am well aware how it works. And I stand by my statement that using cloudflare can slow streams when using emby, that is even documented on the cloudflare site. I won't post after this one. But i'm not quite sure you understand how it works.. You say its NOT a proxy, but it's clearly documented that cloud flare IS a reverse proxy. you said in post #7 "They aren't caching pages, they're serving requests locally." its also documented they ARE caching HTML, JS, Images, CSS (pages) and are serving it locally from edge servers. Again, I'm only pointing out that we don't have control of cloudflares bandwidth for streams and it can cause issues. I still love you chiefnerd... But one thing you CAN teach me... I've never heard "you're simply teaching a Granny how to suck eggs" lol I assume it means i'm wasting my time.. what does it mean? Link to comment Share on other sites More sharing options...
chiefnerd 18 Posted January 19, 2017 Author Share Posted January 19, 2017 Do you think you've got control of your own bandwidth mate? Unless your tier 1 dedicated you have zero control of your bandwidth and I'll bank anything on the fact that cloudflare do. Do you understand the difference between a request and actually serving content? Request hits cloudflare, request analysed then split according to proxy rules - locally allowed cached content served, no cacheable sent to origin and packaged in the response. They don't give the video, they simply defer the request to origin direct. It isn't played via their servers, they're merely deferring ie Proxying. Link to comment Share on other sites More sharing options...
pir8radio 1301 Posted January 19, 2017 Share Posted January 19, 2017 Ok, I will respond one more time The streamed content does not come directly from the origin to the client it runs through the cloudflare servers as i said before.. here is me watching a movie.. this is a .TS file which is an emby stream.. Notice the headers info in yellow thats not MY origin server... you can do a trace to cloudflare on that one... Also the server header is clearly cloudflare-nginx I mean its in black and white. I wish it did go directly to the origin I would be happy then lol.. Link to comment Share on other sites More sharing options...
pir8radio 1301 Posted January 19, 2017 Share Posted January 19, 2017 (edited) THis is probably easier to read. All video streams DO go through cloudflare. Edited January 19, 2017 by pir8radio Link to comment Share on other sites More sharing options...
chiefnerd 18 Posted January 19, 2017 Author Share Posted January 19, 2017 Bangs his head off the table. Obviously no clue how proxying works and I've given up explaining - check your own server logs and if you have the correct headers set you'll see the requests and serves. If you don't have correct headers as in $realip it'll show cloudflare addresses instead of actual request origin IP. Setting up a local proxy correctly is a different topic, as is configuring your web server for cloudflare headers and nothing to do with this thread in the slightest. Link to comment Share on other sites More sharing options...
pir8radio 1301 Posted January 19, 2017 Share Posted January 19, 2017 (edited) ugh... lol we are both getting aggravated. I don't question $realip that just relays the pre-proxy host address through the proxy to my nginx and again to the application server (emby). Yes emby and nginx shows my actual user host IP's not the proxy... My point, and the above screen shot is from the remote browser client, it is receiving the VIDEO stream from cloudflare servers not directly from origin as you said, to my original point the video streams MUST pass through the cloudflare servers. I'm done.. Well, lets agree to disagree buddy... we don't need any more stress in our day. I slightly enjoyed the chess game. Edited January 19, 2017 by pir8radio Link to comment Share on other sites More sharing options...
pir8radio 1301 Posted January 19, 2017 Share Posted January 19, 2017 I still don't know what granny sucking eggs means.... Link to comment Share on other sites More sharing options...
Spaceboy 2557 Posted January 20, 2017 Share Posted January 20, 2017 It means you are telling someone about something that they already know. Now the etymology of it? No idea 1 Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now