RSS
 

Archive for the ‘Technical’ Category

Data layer: Model, Entity, DTO, and DAO

04 Sep

There’s confusion and ambiguity in data layer terminology, especially if you are not in the habit of defining data layers of an application from the ground up.  Let’s try to clear this up. Why is this important? Terminology is important to ensure a common understanding and communication of concepts so that the team is on the same track as it evolves and interacts with the data. Getting data wrong leads to bugs and inefficiencies in the application. Maintaining a clear abstract definition of data definition and organization helps to mitigate such problems.

There is often a disparity between the concrete representation and storage of data from its necessary use within the application. Let’s break down each concept by how they work together to create a data layer segregated into sub-layers with specific responsibilities.

Concept Layer Primary Purpose Key Trait
Entity Persistence Mirror DB structure Contains ORM mappings (e.g., @Entity)
Model Domain/Business Enforce business rules and logic The core “truth” of your application
DTO Service/Presentation Transfer data across process boundaries A simple data bag with no logic
DAO Persistence Abstract DB access operations Provides CRUD methods for an Entity

So, to get terminology correct, we will explain the difference between an “entity” and “model” and how DTO and DAO play roles. The core difference is one of concern and context: an Entity is about the database representation, while a Model is about the business logic representation.

Entity: The Database Ambassador

An Entity is a class that is directly mapped to a database table. Its purpose is to represent the state and structure of a database record.

  • Primary Concern: Data Persistence

  • Lives in: The Data Access/Persistence Layer

  • Characteristics:

    • Each instance corresponds to a row in a table.

    • Its fields/properties map directly to table columns.

    • It often includes ORM (Object-Relational Mapping) annotations like @Entity@Table@Column (in Java/JPA) or similar decorators in other languages (e.g., [Table] in C#, @entity in Python with SQLAlchemy).

    • It defines relationships (e.g., @OneToMany@ManyToOne) to other entities.

  • Example: A UserEntity class with an idusernamepassword_hashcreated_date, etc., that exactly matches the users database table.

    // Java (JPA) Example
    @Entity
    @Table(name = "users")
    public class UserEntity {
        @Id
        @GeneratedValue(strategy = GenerationType.IDENTITY)
        private Long id;
    
        @Column(name = "username", nullable = false, unique = true)
        private String username;
    
        @Column(name = "password_hash", nullable = false)
        private String passwordHash;
    
        // ORM Annotations, getters, and setters
    }

Model (aka Domain Model): The Business Brain

Model (also often called a Domain Model or Business Model) is a class that represents a core concept of your business domain. It contains both data and the behavior (business logic) that operates on that data.

  • Primary Concern: Implementing Business Logic and Rules.

  • Lives in: The Domain Layer (the heart of the application).  It is persistence-ignorant (no ORM annotations) and rich with business methods.

  • Characteristics:

    • It is persistence-ignorant; it has no knowledge of how or where its data is saved. It should not have any ORM annotations.

    • Encapsulates the business rules and logic of your application — methods that enforce business rules (e.g., calculateTotal()validate()changePassword()).

    • It is the most important layer and should be the least likely to change if your database or API changes.

  • Example: A User model that has a method changePassword() which enforces password complexity rules before updating its own password field.

    // Java Example - Pure Domain Model
    public class User {
        private Long id;
        private String username;
        private String password; // Might be hashed by a service, not the model itself
    
        // Business logic lives here
        public boolean isValidPassword(String inputPassword) {
            // Logic to validate password against stored hash
            return PasswordEncoder.matches(inputPassword, this.password);
        }
    
        // Getters and setters might have validation logic
        public void setUsername(String username) {
            if (username == null || username.trim().isEmpty()) {
                throw new IllegalArgumentException("Username cannot be empty");
            }
            this.username = username;
        }
        // ... other getters and setters
    }

The Model is the “truth” of your business. The Entity is just a persistence mechanism for that model. In simpler applications, the Entity and the Model are often the same class, but this is considered an anti-pattern in complex domains because it couples your business logic to your database schema.

Data Transfer Object (DTO): The Data Courier

DTO is a simple container for data used to move information between layers, especially across network boundaries. Its sole purpose is to reduce the number of method calls by aggregating data.

  • Primary Concern: Data Transfer, often over a network (e.g., API requests/responses).

  • Lives in: The Service Layer or Presentation Layer.

  • Characteristics:

    • It contains no business logic—only fields, getters, and setters.

    • It is often tailored to a specific use case (e.g., UserCreationRequestUserSummaryResponse).

    • Provides a stable, decoupled contract for external communication; used to decouple your internal domain/entity from your external API contract.

    • It protects your API from internal changes. You can change your User model’s structure without breaking your API clients, and vice-versa.

  • Example: A UserDto sent to a web client that excludes sensitive fields like passwordHash and includes a calculated fullName field.

    // Example DTO for sending user data to a client
    public class UserDto {
        private Long id;
        private String username;
        private String email;
        private String fullName; // A field that might not exist in the Entity
    
        // No logic, only getters and setters
        public String getFullName() {
            return fullName;
        }
        public void setFullName(String fullName) {
            this.fullName = fullName;
        }
    }

Data Access Object (DAO)

DAO is a design pattern that provides an abstract interface to the database. It hides all the intricate details of the persistence mechanism (e.g., SQL, NoSQL queries) from the rest of the application.

  • Primary Concern: Abstracting data access and persistence operations.

  • Lives in: The Data Access Layer.

  • Characteristics:

    • Separates persistence logic from business logic.
    • It provides CRUD (Create, Read, Update, Delete) operations for a specific entity/model.

    • The application uses the DAO interface, not direct database calls. This makes it easy to switch databases (e.g., from MySQL to PostgreSQL) or switch from SQL to a web API.

    • In modern development, the DAO is often replaced or implemented by an ORM (like Hibernate, Entity Framework, Django ORM, Sequelize). The Repository pattern is a more advanced evolution of the DAO pattern.

  • Example: A UserDao interface with methods like findByIdsavedelete, and findAll.

    // DAO Interface
    public interface UserDao {
        Optional<UserEntity> findById(Long id);
        List<UserEntity> findAll();
        UserEntity save(UserEntity user);
        void delete(Long id);
    }
    
    // JPA-based implementation of the DAO interface
    @Repository // Spring stereotype marking this as a DAO
    public class JpaUserDao implements UserDao {
    
        @PersistenceContext
        private EntityManager entityManager;
    
        @Override
        public Optional<UserEntity> findById(Long id) {
            return Optional.ofNullable(entityManager.find(UserEntity.class, id));
        }
        // ... implement other methods
    }

How They Work Together in a Flow

Let’s imagine an API endpoint GET /users/{id}:

  1. Presentation Layer (Controller):

    • Receives the HTTP request and extracts the user id.

    • Calls a method on a Service.

  2. Service Layer:

    • The service contains business logic. It calls a DAO to fetch the data.

  3. Data Access Layer (DAO):

    • The DAO (e.g., userDao.findById(id)) executes the database query.

    • It returns a UserEntity object, which is a direct representation of the database row.

  4. Service Layer (cont.):

    • The service might perform operations using the User model (if one exists). It might convert the UserEntity to a User domain model to apply business rules.

    • It then converts the result (either the UserEntity or the User model) into a UserDTO tailored for the response (e.g., hiding the password hash, formatting names).

  5. Presentation Layer (Controller):

    • The controller receives the UserDTO from the service and serializes it to JSON to send back in the HTTP response.

This flow ensures each layer has a single responsibility and is decoupled from the others, leading to maintainable, testable, and flexible software.

DTOs and Models: A Flexible Relationship

DTOs closely parallel (and may even match) the corresponding data model

  • Often True, but Not a Rule: It’s very common for a DTO to initially look identical to the domain model it represents, especially for simple CRUD operations. A UserModel might have idname, email, and the UserResponseDto might have those same fields.

  • The Divergence is the Point: The power of the DTO pattern emerges when they stop matching. DTOs are tailored to the specific needs of a client or use case, while the Model is the single source of truth.

    • Combination: A UserProfileDto might combine data from a UserModel and a UserProfileModel.

    • Omission: A PublicUserDto might return a user’s name and avatarUrl but omit their email for privacy.

    • Addition: A UserDto might include a calculated field like isEligibleForDiscount that isn’t stored in the model but is derived from its data.

    • Flattening: A UserOrderDto might flatten a relationship, taking the orderNumber from an associated OrderModel instead of nesting the entire OrderModel object.

DTOs often start life as mirrors of Models but are expected to diverge based on external requirements. The Model is stable; the DTO is volatile and adaptable.

DAOs and Entities: A Direct Mirror

DAOs closely match the entity definitions

  • This is Almost Always True: This relationship is much more direct and rigid. A DAO’s single responsibility is to perform CRUD operations on a specific Entity.

  • One-to-One Mapping: You will typically have:

    • UserEntity class.

    • UserDao interface (e.g., UserRepository in Spring).

    • The UserDao will have methods like save(UserEntity entity)findUserEntityById(Long id), and delete(UserEntity entity).

  • The DAO is Contracted to the Entity: The DAO’s interface is defined by the structure and relationships of the Entity it manages. Its purpose is to translate that object-oriented structure into persistence commands (SQL, etc.).

The DAO is fundamentally tied to its corresponding Entity. This is a very strong, one-to-one coupling by design.

Visualizing the Relationships

A Practical Example

Imagine an application with a User and a Order model.

  1. Persistence Layer:

    • UserEntity (id, username, password_hash, created_date)

    • OrderEntity (id, user_id, total_amount, status)

    • UserDao – methods like save(UserEntity)findById(id)

    • OrderDao – methods like save(OrderEntity)findOrdersByUser(UserEntity)

    Here, the DAOs are tightly coupled to their respective Entities.

  2. Business Layer:

    • User model (id, username, password, ordersList, hasPasswordExpired())

    • Order model (id, totalAmount, status, canBeCancelled())

  3. API Response:

    • You need an API endpoint that returns a user’s profile along with a summary of their recent orders.

    • You would create a UserProfileDto:

      public class UserProfileDto {
          private Long id;
          private String username;
          private List<OrderSummaryDto> recentOrders; // <- Flattened/combined data
      }
    • This DTO does not match any single Model. It combines data from the User model and the Order model and presents it in a tailored structure for the client.

Final Takeaway: Your intuition is spot-on for the data access layer (DAO/Entity mirroring). For the interface layer (DTO/Model), the parallelism is a common starting point, but the strategic value of DTOs is realized when they are allowed to be different from the domain models to meet external demands without polluting the core business logic.

 
 

Data Transfer Objects and NextJS Server Actions

01 Aug

Next.js server actions can be leveraged to implement the DTO (Data Transfer Object) design pattern to structure and manage data flow between the client and server.

What Are Data Transfer Objects (DTO)s?

DTOs are objects that define the structure of data being transferred between layers of an application (e.g., client and server). They are typically used to:

  1. Encapsulate Data: Define a clear contract for the shape of data being sent or received.
  2. Validate Data: Ensure that the data conforms to a specific structure.
  3. Decouple Layers: Abstract the internal structure of the server or database from the client.

DTOs with Next.js Server Actions

Server actions in Next.js provide a way to handle server-side logic directly in the application. By using DTOs as interfaces for server actions, you can:

  1. Standardize Data Contracts
    • Define clear and reusable interfaces for the data sent to and received from server actions.
    • Ensure consistency across the application.
  2. Improve Type Safety
    • Use TypeScript to enforce the structure of data at compile time.
    • Reduce runtime errors caused by unexpected data shapes.
  3. Simplify Validation
    • Validate incoming and outgoing data against the DTO structure.
    • Use libraries like zod or class-validator for runtime validation.
  4. Enhance Maintainability:
    • Centralize data definitions, making it easier to update and refactor the application.

How to Implement DTOs with Server Actions in Next.js

Step 1: Define DTO Interfaces

Create TypeScript interfaces or types to define the structure of the data being transferred.

import { z } from "zod";

// Define the DTO schema using zod for runtime validation
export const SheetUserWeightsDTO = z.object({
  userId: z.string(),
  sheetId: z.string(),
  specWeights: z.record(z.string(), z.number()), // Example: { spec1: 0.5, spec2: 0.8 }
  specsVersion: z.number(),
});

// Infer the TypeScript type from the zod schema
export type SheetUserWeightsDTOType = z.infer<typeof SheetUserWeightsDTO>;

Step 2: Use DTOs in Server Actions

Leverage the DTOs in server actions to validate and enforce the structure of incoming and outgoing data.

import { SheetUserWeightsDTO, SheetUserWeightsDTOType } from "@/lib/dto/sheetUserWeights.dto";

export async function saveSheetUserWeightsAction(data: SheetUserWeightsDTOType): Promise<void> {
  // Validate the incoming data
  const parsedData = SheetUserWeightsDTO.parse(data);

  // Perform the server-side logic (e.g., save to database)
  const { userId, sheetId, specWeights, specsVersion } = parsedData;

  // Example: Save to database
  await db.collection("sheetUserWeights").updateOne(
    { userId, sheetId },
    { $set: { specWeights, specsVersion } },
    { upsert: true }
  );
}

export async function getSheetUserWeightsAction(userId: string, sheetId: string): Promise<SheetUserWeightsDTOType | null> {
  // Fetch data from the database
  const result = await db.collection("sheetUserWeights").findOne({ userId, sheetId });

  if (!result) return null;

  // Validate the fetched data
  return SheetUserWeightsDTO.parse(result);
}

Step 3: Use DTOs in Client-Side Hooks

Use the DTOs in React Query hooks to ensure type safety and consistency.

import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
import { SheetUserWeightsDTOType } from "@/lib/dto/sheetUserWeights.dto";
import { getSheetUserWeightsAction, saveSheetUserWeightsAction } from "@/lib/actions/sheets";

export function useQuerySheetUserWeights(userId: string, sheetId: string) {
  return useQuery<SheetUserWeightsDTOType | null, Error>({
    queryKey: ["sheetUserWeights", userId, sheetId],
    queryFn: () => getSheetUserWeightsAction(userId, sheetId),
    enabled: !!userId && !!sheetId,
    staleTime: 1000 * 60 * 5, // Cache for 5 minutes
  });
}

export function useSaveSheetUserWeights() {
  const queryClient = useQueryClient();

  return useMutation<void, Error, SheetUserWeightsDTOType>({
    mutationFn: saveSheetUserWeightsAction,
    onSuccess: (_, variables) => {
      queryClient.invalidateQueries(["sheetUserWeights", variables.userId, variables.sheetId]);
    },
  });
}

Benefits of This Approach

  1. Type Safety
    • Both the client and server share the same DTO definitions, ensuring consistent data structures.
  2. Validation:
    • DTOs can validate incoming and outgoing data at runtime, catching errors early.
  3. Reusability:
    • DTOs can be reused across server actions, client-side hooks, and even database queries.
  4. Scalability:
    • As the application grows, DTOs provide a clear and maintainable way to manage data contracts.

When to Use DTOs

  • Complex Applications:
    • DTOs are especially useful in applications with complex data flows or multiple layers (e.g., client, server, database).
  • Shared Data Contracts:
    • When the same data structure is used across multiple parts of the application (e.g., client and server).
  • Validation Requirements:
    • When you need to validate data at runtime to ensure correctness.

Conclusion

Using DTOs as interfaces for server actions in Next.js is a powerful design pattern that improves type safety, validation, and maintainability. By defining DTOs with tools like zod or TypeScript interfaces, you can standardize data contracts across your application and ensure consistency between the client and server.

This article was generated mostly via my prompting of AI CoPilot; Image was generated via Claude.ai. I had to edit the result.

 
 

Adding Markdown (mdx) Support to NextJS with App Router

04 Jul

Markdown allows text content to be written with implicit formatting that nearly matches how one would write text without thought of formatting. A lot of web content is also simply text content; a lot of web content is built on the ReactJS library; and NextJS has become a popular framework, extending ReactJS. With that, I was motivated to use the Markdown content of my NextJS web app, but I had trouble simply relying on their documentation, so I thought I would document what I got working.

  1. Use Markdown for a text-heavy portion of a page of my web app, and
  2. To be able to have an entire page of content come from a single Markdown page

The former is a feature of NextJS to allow markdown content to be imported as a “component,” which can be rendered like any component in ReactJs.

The latter would rely NextJS’s App Router that assumes many URL paths for the app parallel the file/directory hierarchy of the app itself. So, merely placing a “page” file in the app’s directory will create a new URL path for the website. That means that you can create a “page.md” or “page.mdx” file with Markdown text content to define the content for that URL. .mdx files also allow JSX syntax so that React/Next components can be embedded within the Markdown. Read the rest of this entry »

 
 

Command line Clipboard for macOS, Cygwin, and Linux

25 Mar

I use the command line a lot, even though I am on a graphical user-interface (GUI) on Windows, macOS, or Linux. And since I’m lazy, I write a lot of scripts to perform repetitive tasks. I stumbled across macOS commands that allow command line programs to copy/paste between the clipboard that we’re so used to using.

macOS pbpaste and pbcopy

macOS has two commands, pbpaste and pbcopy, which “paste” from the pasteboard to stdout and copies from stdin to the pasteboard, respectively. Read the rest of this entry »

 

ChatGPT: What is the common way to parse command line parameters with Rust?

04 Mar

Rust Programming Language logoIn Rust, the most common way to parse command-line parameters is by using the std::env::args function, which returns an iterator over the command-line arguments passed to the program. Read the rest of this entry »

 

Ruby RSpec FactoryBot traits and transient via ChatGPT

15 Dec

The following was edited together from a series of questions to ChatGPT on the topic. Currently, I cannot identify sources of the originating content. My role was to edit ChatGPT into the following.

FactoryBot

RSpec is a testing framework for the Ruby programming language, and FactoryBot (formerly known as Factory Girl) is a library for creating test data in Ruby. Together, these tools can be used to write unit-tests for a Ruby application.

In FactoryBot, a factory is a blueprint for creating test data objects. A factory can define various attributes of the objects it creates, such as the object’s type, the values of its attributes, and any associations it has with other objects.

Read the rest of this entry »
 
 

Frontend Security Recipe Checklist

03 Jan

Even as the number of frontend programming vulnerabilities grows continually, many are not difficult to combat; you simply need to remember to fortify your frontend security against them.

 
 

ReactJS: componentWillReceiveProps() is Dead! (*sniff*)

21 Apr

ReactJSThings evolve. ReactJS evolves. With version 16.3, there were several changes to the component life-cycle methods. In particular, componentWillReceiveProps is disappearing. In its place, they say, you can use the getDerivedStateFromProps static function. I found this a bit challenging, but I did find an interesting pattern when the component-state was dependent on fetched information.

I should mention that I had a specific goal to better encapsulate data within the component. While I could pass in all the needed data as properties, that would require the surrounding component know what to pass and how to get it. That shouldn’t necessarily be necessary; the component knows what it needs and how to retrieve it.

For example, say you have a component which accepts a phone number and displays the phone number and the state that it’s from. Certainly, you could write a simple component that accepts both pieces of information as properties.

<ShowPhoneLocation number="+12065551212" city="Seattle" />

Which might be implemented as:

class ShowPhoneLocation extends React.Component {
  render() {
    return (
      <div>{this.props.number} is from {this.props.city}</div>
    )
  } // render()
} // class ShowPhoneLocation

But, since the component should be able to infer the state from the phone number (by its area code), it shouldn’t be incumbent on its container to know what it is.

class ShowPhoneLocation extends React.Component {
  static getDerivedStateFromProps(nextProps, prevState) {
    let location = getCityFromPhone(nextProps.number)
    return {
      city: location
    }
  }
  render() {
    return (
      <div>{this.props.number} is from {this.state.city}</div>
    )
  } // render()
} // class ShowPhoneLocation

That’s all well and good, but what if getCityFromPhone() has to call a web service? We don’t want getDerivedStateFromProps() to stall, waiting for a response. However, it is static and does not have a this reference to the object for which it is returning state; so an asynchronous fetch doesn’t know what object’s state to set. Instead, don’t wait for the result to save in the state, save the request’s Promise in the state and update the state, once the promise resolves.

function getCityFromPhone(number) {
  return fetch('http://saas.com/get/'+number+'/city') // Returns fetch promise
}
class ShowPhoneLocation extends React.Component {
  static getDerivedStateFromProps(nextProps, prevState) {
    let location = getCityFromPhone(nextProps.number)
    return {
      city: location
    }
  }
  componentDidUpdate() {
    let location = this.state.city
    if (location instanceof Promise) {
      this.setState({ city: '...waiting...' })
      location.then(city => this.setState({ city: city }) )
        .catch(() => this.setState({ city: 'not found' }) )
    }
  }
  render() {
    return (
      <div>
        {this.props.number} is from {this.state.city instanceof Promise
         ? '...'
        : this.state.city}</div>
    )
  } // render()
} // class ShowPhoneLocation

In componentDidUpdate() you can define the completion handlers to set the object’s state, base on the returned information from the service.

It is a common pattern to perform a fetch in componentDidMount(). The problem is that there may not be enough information to perform the fetch, that early, or the information for the fetch changes after the component has been mounted.

I am going to miss componentWillReceiveProps()… without it, things become a bit more convoluted but it’s going the way of the Dodo.

 

HTTP via Telnet

02 May

Okay, back to basics, the very low level basics. HTTP is text based, telnet enables a network connection via a text-based command line. All good. We can use telnet to talk to a web server by manually typing text to it, if you know the details of the content syntax to send HTTP via telnet.

Though this is not done, often, it is still good to know how it can be done. In fact, since these basics rarely change, I shouldn’t have even had to write this down, since it should be innate.

Sending HTTP Using Telnet

Once you get telnet running (see section, below), it is just a matter of typing HTTP to the server, manually.

There are four pieces the first three are shown, here:

GET / HTTP/1.1
Host: cachecrew.com
Connection: close
  1. This is the start of the HTTP commands you can enter. The first line must specify the HTTP method. There are three parts: Read the rest of this entry »
 
 

Promises, Promises… Timeout!

25 Apr

I was playing with ES6’s new Promises, for the first time, this week. Then, I was looking at the ugliness of using a browser’s setTimeout() function and thought that it would look better as a Promise.

tl;dr summary: A Simple Promise Version of “setTimeout()”

If we do it right, you simply specify the timeout period and implement a handler for then() to invoke:

timeout(5000) // Delay for 5000 ms
   .then(function () {
      // Do, here, whatever should happen when the time has elapsed…
   });

Or, since this is ES6, we might as well use arrow-function shorthand:

timeout(5000).then( () => {
   // do your thing, here…
})

Implementation

The setTimeout() already functions a bit like a Promise, without matching the promise-pattern and syntax, so converting it to a Promise is pretty easy: Read the rest of this entry »