Tried doing a ping test.
PING google.com (173.194.46.101): 56 data bytes
64 bytes from 173.194.46.101: icmp_seq=0 ttl=55 time=19.275 ms
64 bytes from 173.194.46.101: icmp_seq=1 ttl=55 time=23.205 ms
64 bytes from 173.194.46.101: icmp_seq=2 ttl=55 time=22.159 ms
64 bytes from 173.194.46.101: icmp_seq=3 ttl=55 time=21.412 ms
64 bytes from 173.194.46.101: icmp_seq=4 ttl=55 time=23.578 ms
64 bytes from 173.194.46.101: icmp_seq=5 ttl=55 time=21.999 ms
64 bytes from 173.194.46.101: icmp_seq=6 ttl=55 time=19.256 ms
64 bytes from 173.194.46.101: icmp_seq=7 ttl=55 time=20.314 ms
64 bytes from 173.194.46.101: icmp_seq=8 ttl=55 time=25.496 ms
64 bytes from 173.194.46.101: icmp_seq=9 ttl=55 time=30.854 ms
64 bytes from 173.194.46.101: icmp_seq=10 ttl=55 time=20.307 ms
64 bytes from 173.194.46.101: icmp_seq=11 ttl=55 time=19.156 ms
64 bytes from 173.194.46.101: icmp_seq=12 ttl=55 time=23.351 ms
64 bytes from 173.194.46.101: icmp_seq=13 ttl=55 time=26.362 ms
64 bytes from 173.194.46.101: icmp_seq=14 ttl=55 time=21.010 ms
64 bytes from 173.194.46.101: icmp_seq=15 ttl=55 time=19.286 ms
64 bytes from 173.194.46.101: icmp_seq=16 ttl=55 time=21.431 ms
64 bytes from 173.194.46.101: icmp_seq=17 ttl=55 time=69.867 ms
64 bytes from 173.194.46.101: icmp_seq=18 ttl=55 time=104.603 ms
64 bytes from 173.194.46.101: icmp_seq=19 ttl=55 time=90.473 ms
64 bytes from 173.194.46.101: icmp_seq=20 ttl=55 time=63.456 ms
64 bytes from 173.194.46.101: icmp_seq=21 ttl=55 time=94.455 ms
64 bytes from 173.194.46.101: icmp_seq=22 ttl=55 time=175.665 ms
64 bytes from 173.194.46.101: icmp_seq=23 ttl=55 time=38.482 ms
64 bytes from 173.194.46.101: icmp_seq=24 ttl=55 time=127.997 ms
Request timeout for icmp_seq 25
64 bytes from 173.194.46.101: icmp_seq=26 ttl=55 time=201.320 ms
64 bytes from 173.194.46.101: icmp_seq=27 ttl=55 time=188.838 ms
64 bytes from 173.194.46.101: icmp_seq=28 ttl=55 time=20.023 ms
64 bytes from 173.194.46.101: icmp_seq=29 ttl=55 time=22.743 ms
64 bytes from 173.194.46.101: icmp_seq=30 ttl=55 time=24.211 ms
64 bytes from 173.194.46.101: icmp_seq=31 ttl=55 time=23.260 ms
64 bytes from 173.194.46.101: icmp_seq=32 ttl=55 time=26.829 ms
64 bytes from 173.194.46.101: icmp_seq=33 ttl=55 time=31.872 ms
64 bytes from 173.194.46.101: icmp_seq=34 ttl=55 time=22.535 ms
64 bytes from 173.194.46.101: icmp_seq=35 ttl=55 time=21.977 ms
64 bytes from 173.194.46.101: icmp_seq=36 ttl=55 time=22.050 ms
64 bytes from 173.194.46.101: icmp_seq=37 ttl=55 time=34.901 ms
64 bytes from 173.194.46.101: icmp_seq=38 ttl=55 time=21.848 ms
64 bytes from 173.194.46.101: icmp_seq=39 ttl=55 time=21.580 ms
64 bytes from 173.194.46.101: icmp_seq=40 ttl=55 time=27.690 ms
64 bytes from 173.194.46.101: icmp_seq=41 ttl=55 time=20.924 ms
64 bytes from 173.194.46.101: icmp_seq=42 ttl=55 time=34.872 ms
64 bytes from 173.194.46.101: icmp_seq=43 ttl=55 time=33.558 ms
64 bytes from 173.194.46.101: icmp_seq=44 ttl=55 time=20.681 ms
Request timeout for icmp_seq 45
64 bytes from 173.194.46.101: icmp_seq=46 ttl=55 time=21.278 ms
64 bytes from 173.194.46.101: icmp_seq=47 ttl=55 time=19.107 ms
64 bytes from 173.194.46.101: icmp_seq=48 ttl=55 time=28.842 ms
64 bytes from 173.194.46.101: icmp_seq=49 ttl=55 time=23.332 ms
Request timeout for icmp_seq 50
64 bytes from 173.194.46.101: icmp_seq=51 ttl=55 time=304.836 ms
Request timeout for icmp_seq 52
64 bytes from 173.194.46.101: icmp_seq=53 ttl=55 time=239.826 ms
64 bytes from 173.194.46.101: icmp_seq=54 ttl=55 time=21.518 ms
64 bytes from 173.194.46.101: icmp_seq=55 ttl=55 time=26.636 ms
64 bytes from 173.194.46.101: icmp_seq=56 ttl=55 time=22.842 ms
Request timeout for icmp_seq 57
64 bytes from 173.194.46.101: icmp_seq=58 ttl=55 time=25.674 ms
64 bytes from 173.194.46.101: icmp_seq=59 ttl=55 time=30.675 ms
Request timeout for icmp_seq 60
64 bytes from 173.194.46.101: icmp_seq=61 ttl=55 time=20.920 ms
64 bytes from 173.194.46.101: icmp_seq=62 ttl=55 time=22.432 ms
64 bytes from 173.194.46.101: icmp_seq=63 ttl=55 time=24.743 ms
64 bytes from 173.194.46.101: icmp_seq=64 ttl=55 time=25.508 ms

We've also tried speedtest. We've gotten results like:
0.03 mb/s download, 0.01 mb/s upload
0.13 mb/s download, 6.32 mb/s upload
25.2 mb/s download, 0.34 mb/s upload

It's ALL over the place and makes no sense. I tried a different router, I've replaced all the cables going into the modem and the router. I'm pretty sure this is comcast's end of things, but if anyone has any suggestions, I would love to hear it.