TL;DR: Kafka Streams Scala Application: sbt new monksy/kafka-streams.g8
I’ve been working on a new project, which I’ll give more details on later, that involves a few Kafka Streams applications. As with any new Scala project, you should use a G8 template to start out with it. Starting out with a premade template is just a good practice. Additionally, templates can do a lot of work for you when it comes to good practices, structures, and extra compilation tools.
Unfortunately, with the Confluence/Kafka world, there aren’t many Kafka-Streams Scala based templates. On Github I found two that came up in the search:
The sv3ndk template used the Lightbend version of the kafka-streams-scala. It’s very out of date. So that was out. The idarlington template used Kafka Streams 2.0. Not ideal, but not unworkable. So I forked it.
What was done:
Upgrade the Kafka version from 2 to 2.5 (Latest Release)
Upgrade the testing utilties to use the non-depricated (post 2.4) functionality
Improved the file layout
Added Assembly support
Added a larger gitingore file
Upgraded the project to Scala 2.13
Upgraded the other libraries in the project
Added dependency tree plugin support
Upgraded the g8 build file
What did I learn about?
I learned a lot about creating a G8 template and how the variables are substituted. The existing project had a lot of this work already done, however, I did have to do some of my own substitutions.
MergeStrategies and dealing with “module-info.class” in the assembly plugin. (Hint: Merge strategy rule for: case “module-info.class” => MergeStrategy.discard) The module-info.class is a new addition that came from Java 9’s Jigsaw to define JVM modules. They’ll pop up in the Jackson libraries.
Some of the built-in Kafka Streams Test utilities. For the most part, I’ve been using the mocked streams library.
G8 Sbt plugin. Use sbt g8 to build an example copy of the application in target/g8. From there it’s a lot easier to test and build up your template. Use sbt g8Test to run an automated test. I’m not sure how to customize the sbt tasks. I’m sure it’s a configuration option.
I’ve seen a lot of documentation for checking if a variable is set by calling [[ !-z "$VARNAME" ]]. That condictional statement essentially checks to see if the resolved string "$VARNAME" is not an empty string. Not very straight forward is it?
However, I found there is a new (well to me) conditional statement to confirm that the variable is set without all of this. It’s -v.
From the documentation:
True if the shell variable varname is set (has been assigned a value).
Due to the social distancing orders. We’re now not expected to gather. This makes it harder to play games.
Jackbox Games – These are games that are played by sharing a screen and each player uses their mobile to complete actions within the game. (This is the go-to set of games for hosting) Video conferencing is used to show the game direction. There are many game types in the bundles. My favorite is TKO. Where to acquire: https://www.jackboxgames.com/games/
Spyfall- This is a deduction game, similar to Clue, where you’re responsible for finding out who is the spy and the spy is responsible for finding out where everyone is. Video conferencing is used for the discussion of the game. Source: Spyfall.io also Netgames
Secret Hitler- This is a good vs bad teams based game in which there are different abilities and you have to call out the bad team. Video conferencing is used for the discussion portion of the game. Netgames and SecretHitler.io
One Night Ultimate Werewolf- This is a game of accusations and avoidance of werewolves. This is the moderator’s less version of the game. The video conferencing system is used for the discussion and accusation of the roles. Netgames
Love Letter- This is a game of role/classes, strategic interference, and abilities. It’s a quick learning game where strategy puts you to an advantage. Video conferencing is used for interaction in the game. Netgames, Board Games ArenaSteam/Multiplayer Online
Ticket to Ride- This is a board game where you’re trying to build the greatest empire via valuable routes. The android and steam version of this allows you to play online. This is also a pass and play game if you’re playing with a lot of people in the same room. Google Play / Apple Store
Trivia – This is a little harder to play since it requires a moderator and keeping score. At the moment none of the video conferencing options allow you to have breakout teams that allow you to escalate a message to the top. Until then you’ll have to play one player/room as a team. Also there’s no guarantee that people won’t cheat. I would suggest buying: University Games
Have other games that would qualify for this? Please leave it in the comments down below.
I kind of shocked my friends a week ago by creating a blog post. It’s been a while since I’ve written a post. I’ve been quite busy in my work life, and I just didn’t prioritize writing anything here.
I miss it a bit and and I don’t a little. I enjoy the ability to document and journal here. However, I get really frustrated that the blog isn’t a very good community. (Note that’s a blog post for another day)
To cut this short, I may blog about the following things in the upcoming days: How jCrete was (that was last July), learning ESP, the Kafka Streams example, how I could see businesses adapt to these new rules of the quarantine.
It’s late, but that’s not a bad first attempt. From the hello world source it looks like a fairly straight forward C application. (This is going to take some refreshing, but I love the lean-ness)
What do I hope to accomplish with this? I hope to learn more about the ESP32 embedded systems and I hope to create 2 devices that will go on my HomeAssistant network. One for air quality monitoring, and another for humidity and temperature monitoring (DH11 device). Also I have a few extras for the device (like an e-ink screen that’ll be pretty cool to play arround with)
It’s called the serialization checker. From the readme page it’s here to solve:
The root problem that led to this project’s creation is that REST typically uses JSON, and that JSON is Schemaless. This makes it difficult to create data objects to interact with services. In the case of connecting to a third-party REST service, you typically have lots of examples. This project helps you, the developer, iterate through the creation of the data objects.