I've had the Nexus 5X since it came out. And have still been using my Nexus 5 this whole time. At first I was unsure how I felt about the 5X which I had pre-ordered sight unseen. Now I feel I can give a fuller review and comparison.
I really liked the size of the Nexus 5, the dimensions as well as the weight. The Nexus 5X is actually thinner than the 5, ever so slightly, but significantly taller and noticeably wider when you place the two phones back to back. The 5X also feels bigger in the hand and the difference in length makes it balance differently, sometimes feeling like a bigger or heavier phone, even if it's not. The trend is for phones to get bigger but I was not upgrading with the intent of wanting a bigger phone.
The camera on the Nexus 5X is nicer than the 5 when it comes to image quality and photography in lower light conditions. But there's one big problem. The camera software on the 5X often lags and even stops working. Seemingly depending on the current load and available memory on your phone, the camera can take several seconds before starting to show an image and be ready to snap. This is way too long to be acceptable. On the flip side, the Nexus 5X with Marshmallow has a great new feature where you can directly open the camera app by double-clicking the power button. When this works, you can start taking pictures quicker than most other phones. But when the camera software is lagging, you will be the last person to snap a shot.
Nexus Imprint: Another added hardware feature is the new fingerprint sensor below the camera. I've found this to work almost all of the time, although rarely it will think I'm trying to open it when I'm not touching it. It's also in a good location and detects quickly, and if your phone isn't lagging then you can start using it soon after touching your finger to it. You can add multiple fingerprints.
USB Type C. This may be the future of USB cables. It's still slightly in the future though. It's nice that the cable can be inserted in both orientations, a common frustration with USB cables. The problem is that it's now 2016 and still nobody else is using USB C, and it's hard to go and buy a cheap replacement or an adapter to other types of USB, such as in order to connect your phone to your computer. The Nexus 5X does charge fast with the cable though. I wish Google had included a USB Type C to Type A adapter though.
Display. The display is bigger and this is a big plus as you can fit more on the screen. Any difference in quality is less noticeable except in close side-by-side comparisons where things will look nicer on the Nexus 5X.
Ambient Display: This is yet another nice new feature of the 5X. When you move the phnoe, say picking it up, but without pressing any buttons, the screen will come on in grayscale and display the notifications. It will also turn on in Ambient Display mode when a notification comes in so you can react to it without turning your phone screen on. You can expand notifications or open them directly from there.
Battery life: Not only does the Nexus 5X charge faster than the 5, it will last much longer. Battery life was a common problem with the Nexus 5. My Nexus 5X often has half charge remaining by evening. No anxiety about overusing the device.
CPU and performance. One would expect a 2 year newer upgraded version of a model should be better in every way. Mobile processors have gotten better (all while drawing battery at an acceptable rate). Unfortunately, this is a real disappointment with the Nexus 5X and I'm not sure why LG allowed this. Not only does it not feel noticeably speedier for most tasks, the phone has a problem which causes it to lag. Many others have complained about this and solutions (temporary) range from rebooting, to clearing caches, to running in safe mode and avoiding certain 3rd party (non-Google) apps. When the phone is lagging, it will take several seconds to register a touch response and even physical button (power, volume) press, the camera will take many seconds to load, and you may notice the phone has heated up. One theory is that overheating from CPU activity causes the CPUs to shift down to a lower clock speed. Being out of memory may also be a root cause of the lag.
The end result is a very frustrating experience for a Nexus phone. The problem isn't the added software, it's something inherent to the LG hardware and Android Marshmallow. This device should have been a better Nexus 5 in every way. Unfortunately, I don't think the 5X deserves the fanfare which the original Nexus 5 deserved. Since you are already upgrading to a bigger device than the 5, you should seriously consider the Nexus 6P instead which owners have been loving and which doesn't suffer from any lag problems.